[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:36:28 dist-test-slave-9fn6 krb5kdc[11238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:36:28 dist-test-slave-9fn6 krb5kdc[11238](info): set up 2 sockets
Sep 28 16:36:28 dist-test-slave-9fn6 krb5kdc[11238](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260928 16:36:29.597126 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.007s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:36:29 dist-test-slave-9fn6 krb5kdc[11238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:36:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:36:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:36:29.810374 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37617
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:30.343293 11251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:30.343802 11251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:30.344010 11251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:30.352506 11251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:36:30.352725 11251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:30.352820 11251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:30.352918 11251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:36:30.352988 11251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:36:30.364527 11251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:30.366447 11251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:30.368343 11251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:30.383103 11256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:30.383142 11257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:30.383071 11259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:30.383836 11251 server_base.cc:1034] running on GCE node
I20260928 16:36:30.384974 11251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:30.386713 11251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:30.387949 11251 hybrid_clock.cc:648] HybridClock initialized: now 1790613390387869 us; error 61 us; skew 500 ppm
I20260928 16:36:30.388351 11251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:30 dist-test-slave-9fn6 krb5kdc[11238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613390, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:30.392150 11251 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:36:30.394049 11251 webserver.cc:460] Webserver started at http://127.10.247.254:34289/ using document root <none> and password file <none>
I20260928 16:36:30.395022 11251 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:30.395164 11251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:30.395570 11251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:30.399721 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "bea535aeac0247a285808722fa31fc7b"
format_stamp: "Formatted at 2026-09-28 16:36:30 on dist-test-slave-9fn6"
server_key: "348deaa5a1f292cd81d6d1d706780c84"
server_key_iv: "bc19c38698ebb462b046186a36870376"
server_key_version: "encryptionkey@0"
I20260928 16:36:30.400851 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "bea535aeac0247a285808722fa31fc7b"
format_stamp: "Formatted at 2026-09-28 16:36:30 on dist-test-slave-9fn6"
server_key: "348deaa5a1f292cd81d6d1d706780c84"
server_key_iv: "bc19c38698ebb462b046186a36870376"
server_key_version: "encryptionkey@0"
I20260928 16:36:30.407935 11251 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.005s
I20260928 16:36:30.412582 11267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:30.414456 11251 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260928 16:36:30.414672 11251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "bea535aeac0247a285808722fa31fc7b"
format_stamp: "Formatted at 2026-09-28 16:36:30 on dist-test-slave-9fn6"
server_key: "348deaa5a1f292cd81d6d1d706780c84"
server_key_iv: "bc19c38698ebb462b046186a36870376"
server_key_version: "encryptionkey@0"
I20260928 16:36:30.414911 11251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:30.437230 11251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:30.442392 11251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:30.442754 11251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:30.465149 11251 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:37617
I20260928 16:36:30.465173 11318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:37617 every 8 connection(s)
I20260928 16:36:30.466802 11251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:36:30.468240 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 11251
I20260928 16:36:30.468641 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:36:30.470940 11231 external_mini_cluster.cc:1428] Setting key 1ea7c08f8bd8b8e7abfcfbfd2c5226ae
I20260928 16:36:30.472481 11319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:30.486691 11319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b: Bootstrap starting.
I20260928 16:36:30.490646 11319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:30.491837 11319 log.cc:824] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:30.495373 11319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b: No bootstrap required, opened a new log
Sep 28 16:36:30 dist-test-slave-9fn6 krb5kdc[11238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:30.505380 11319 raft_consensus.cc:348] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea535aeac0247a285808722fa31fc7b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37617 } }
I20260928 16:36:30.505914 11319 raft_consensus.cc:374] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:30.506040 11319 raft_consensus.cc:729] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea535aeac0247a285808722fa31fc7b, State: Initialized, Role: FOLLOWER
I20260928 16:36:30.506862 11319 consensus_queue.cc:260] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea535aeac0247a285808722fa31fc7b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37617 } }
I20260928 16:36:30.507154 11319 raft_consensus.cc:388] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:36:30.507308 11319 raft_consensus.cc:482] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:36:30.507536 11319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:30.510345 11319 raft_consensus.cc:504] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea535aeac0247a285808722fa31fc7b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37617 } }
I20260928 16:36:30.511070 11319 leader_election.cc:302] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bea535aeac0247a285808722fa31fc7b; no voters:
I20260928 16:36:30.511914 11319 leader_election.cc:288] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:36:30.512193 11324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:30.514070 11324 raft_consensus.cc:686] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [term 1 LEADER]: Becoming Leader. State: Replica: bea535aeac0247a285808722fa31fc7b, State: Running, Role: LEADER
I20260928 16:36:30.514444 11321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:30.476012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.10.247.254:37617)
0928 16:36:30.477129 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:30.477148 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:30.478364 (+ 1216us) server_negotiation.cc:407] Connection header received
0928 16:36:30.488287 (+ 9923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:30.488312 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:30.488894 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:30.489302 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:30.492684 (+ 3382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:30.494189 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:30.496120 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:30.496932 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:30.497127 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:30.503745 (+ 6618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:30.503761 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:30.503782 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:30.503844 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:30.507692 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:30.508733 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:30.508744 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:30.508753 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:30.508959 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:30.509518 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:30.509524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:30.509528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:30.510223 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:30.510579 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:30.511693 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0928 16:36:30.512287 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":435,"threads_started":1}
I20260928 16:36:30.515022 11324 consensus_queue.cc:237] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea535aeac0247a285808722fa31fc7b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37617 } }
I20260928 16:36:30.517185 11319 sys_catalog.cc:564] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:36:30.518836 11326 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bea535aeac0247a285808722fa31fc7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea535aeac0247a285808722fa31fc7b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37617 } } }
I20260928 16:36:30.519280 11326 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:30.521277 11325 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bea535aeac0247a285808722fa31fc7b. Latest consensus state: current_term: 1 leader_uuid: "bea535aeac0247a285808722fa31fc7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea535aeac0247a285808722fa31fc7b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37617 } } }
I20260928 16:36:30.521631 11325 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:30.523705 11329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:36:30.530246 11329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:36:30.541850 11329 catalog_manager.cc:1347] Generated new cluster ID: 96f22d8733bf42678376c4e4e0dec05a
I20260928 16:36:30.542018 11329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:36:30.553613 11329 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:36:30.555712 11329 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:36:30.563136 11329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bea535aeac0247a285808722fa31fc7b: Generated new TSK 0
I20260928 16:36:30.564636 11329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:36:30.621306 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37617
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:31.042831 11345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:31.043236 11345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:31.043381 11345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:31.051648 11345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:31.051793 11345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:31.051962 11345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:36:31.064178 11345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37617
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:31.066041 11345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:31.067734 11345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:31.083010 11351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:31.083056 11350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:31.082968 11353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:31.089393 11345 server_base.cc:1034] running on GCE node
I20260928 16:36:31.090230 11345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:31.091501 11345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:31.092710 11345 hybrid_clock.cc:648] HybridClock initialized: now 1790613391092647 us; error 60 us; skew 500 ppm
I20260928 16:36:31.093077 11345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:31 dist-test-slave-9fn6 krb5kdc[11238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:31.096918 11345 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:36:31.098817 11345 webserver.cc:460] Webserver started at http://127.10.247.193:40885/ using document root <none> and password file <none>
I20260928 16:36:31.099701 11345 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:31.099824 11345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:31.100153 11345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:31.103890 11345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "1f6ca65a080140ecbd0dea6e567c6d8c"
format_stamp: "Formatted at 2026-09-28 16:36:31 on dist-test-slave-9fn6"
server_key: "390b82d5d8a17a5e713468773818949d"
server_key_iv: "843b8d32ced7828f546f5ab9f65abebf"
server_key_version: "encryptionkey@0"
I20260928 16:36:31.104846 11345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "1f6ca65a080140ecbd0dea6e567c6d8c"
format_stamp: "Formatted at 2026-09-28 16:36:31 on dist-test-slave-9fn6"
server_key: "390b82d5d8a17a5e713468773818949d"
server_key_iv: "843b8d32ced7828f546f5ab9f65abebf"
server_key_version: "encryptionkey@0"
I20260928 16:36:31.111060 11345 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:36:31.115525 11360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:31.117377 11345 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:36:31.117561 11345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "1f6ca65a080140ecbd0dea6e567c6d8c"
format_stamp: "Formatted at 2026-09-28 16:36:31 on dist-test-slave-9fn6"
server_key: "390b82d5d8a17a5e713468773818949d"
server_key_iv: "843b8d32ced7828f546f5ab9f65abebf"
server_key_version: "encryptionkey@0"
I20260928 16:36:31.117851 11345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:31.135761 11345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:31.140796 11345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:31.141126 11345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:31.142320 11345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:36:31.144079 11345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:31.144167 11345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:31.144340 11345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:31.144403 11345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:31.182610 11345 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:42085
I20260928 16:36:31.182665 11474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:42085 every 8 connection(s)
I20260928 16:36:31.184267 11345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:36:31.185931 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 11345
I20260928 16:36:31.186208 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:36:31.187067 11231 external_mini_cluster.cc:1428] Setting key 1321a8fff28b50745b1e425d1232beb7
Sep 28 16:36:31 dist-test-slave-9fn6 krb5kdc[11238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
I20260928 16:36:31.213369 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:31.187966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:41937 (local address 127.10.247.254:37617)
0928 16:36:31.188948 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:31.188954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:31.189908 (+ 954us) server_negotiation.cc:407] Connection header received
0928 16:36:31.195559 (+ 5651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:31.195565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:31.195662 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:31.195766 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:31.197917 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:31.198746 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:31.201144 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:31.201599 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:31.201729 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:31.206502 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:31.206508 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:31.206512 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:31.206553 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:31.209629 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:31.210494 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:31.210498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:31.210501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:31.210652 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:31.211107 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:31.211111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:31.211113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:31.211533 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:31.211698 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:31.212809 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0928 16:36:31.213040 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":750,"threads_started":1}
I20260928 16:36:31.215306 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:31.188813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37617 (local address 127.10.247.193:41937)
0928 16:36:31.189747 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:31.189796 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:31.195151 (+ 5355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:31.196114 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:31.196129 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:31.196869 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:31.197728 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:31.197752 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:31.198939 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:31.198945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:31.200998 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:31.201014 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:31.201859 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:31.201865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:31.202177 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:31.202997 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:31.203028 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:31.206184 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:31.209802 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:31.209811 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:31.209836 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:31.210351 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:31.210768 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:31.210772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:31.210775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:31.210988 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:31.211688 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:31.211698 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:31.212297 (+ 599us) client_negotiation.cc:769] Sending connection context
0928 16:36:31.212689 (+ 392us) client_negotiation.cc:240] Negotiation successful
0928 16:36:31.213492 (+ 803us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":346,"threads_started":1}
I20260928 16:36:31.217370 11475 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37617
I20260928 16:36:31.217795 11475 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:31.219012 11475 heartbeater.cc:511] Master 127.10.247.254:37617 requested a full tablet report, sending...
I20260928 16:36:31.222231 11284 ts_manager.cc:194] Registered new tserver with Master: 1f6ca65a080140ecbd0dea6e567c6d8c (127.10.247.193:42085)
I20260928 16:36:31.224304 11284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:41937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:36:31.237926 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37617
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:31.657094 11481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:31.657501 11481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:31.657645 11481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:31.665915 11481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:31.666080 11481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:31.666249 11481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:36:31.677713 11481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37617
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:31.679482 11481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:31.681455 11481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:31.697085 11486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:31.697108 11489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:31.697048 11487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:31.698216 11481 server_base.cc:1034] running on GCE node
I20260928 16:36:31.699018 11481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:31.700253 11481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:31.701493 11481 hybrid_clock.cc:648] HybridClock initialized: now 1790613391701405 us; error 74 us; skew 500 ppm
I20260928 16:36:31.701874 11481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:31 dist-test-slave-9fn6 krb5kdc[11238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:31.706080 11481 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:36:31.708209 11481 webserver.cc:460] Webserver started at http://127.10.247.194:46141/ using document root <none> and password file <none>
I20260928 16:36:31.709125 11481 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:31.709255 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:31.709605 11481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:31.713156 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "d10bc4e1a62c4474bf500b1d8249413b"
format_stamp: "Formatted at 2026-09-28 16:36:31 on dist-test-slave-9fn6"
server_key: "4d11bc7469d43e1671f0d4811a45523a"
server_key_iv: "325121c7ba5ee26e36171ce0a6aa7558"
server_key_version: "encryptionkey@0"
I20260928 16:36:31.714071 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "d10bc4e1a62c4474bf500b1d8249413b"
format_stamp: "Formatted at 2026-09-28 16:36:31 on dist-test-slave-9fn6"
server_key: "4d11bc7469d43e1671f0d4811a45523a"
server_key_iv: "325121c7ba5ee26e36171ce0a6aa7558"
server_key_version: "encryptionkey@0"
I20260928 16:36:31.720716 11481 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.006s
I20260928 16:36:31.725419 11496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:31.727350 11481 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:36:31.727573 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "d10bc4e1a62c4474bf500b1d8249413b"
format_stamp: "Formatted at 2026-09-28 16:36:31 on dist-test-slave-9fn6"
server_key: "4d11bc7469d43e1671f0d4811a45523a"
server_key_iv: "325121c7ba5ee26e36171ce0a6aa7558"
server_key_version: "encryptionkey@0"
I20260928 16:36:31.727911 11481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:31.746758 11481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:31.751799 11481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:31.752128 11481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:31.753358 11481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:36:31.755146 11481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:31.755236 11481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:31.755390 11481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:31.755447 11481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:31.794809 11481 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:45295
I20260928 16:36:31.794831 11610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:45295 every 8 connection(s)
I20260928 16:36:31.796437 11481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:36:31.801544 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 11481
I20260928 16:36:31.801827 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:36:31.802495 11231 external_mini_cluster.cc:1428] Setting key 673b965e43fe143c5bdafeab306f7810
Sep 28 16:36:31 dist-test-slave-9fn6 krb5kdc[11238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
I20260928 16:36:31.824132 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:31.799862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:45541 (local address 127.10.247.254:37617)
0928 16:36:31.800366 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:31.800371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:31.801968 (+ 1597us) server_negotiation.cc:407] Connection header received
0928 16:36:31.804556 (+ 2588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:31.804561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:31.804747 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:31.804865 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:31.806974 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:31.808339 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:31.810678 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:31.811401 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:31.811515 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:31.816278 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:31.816285 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:31.816288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:31.816333 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:31.819360 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:31.820644 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:31.820649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:31.820652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:31.820807 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:31.821409 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:31.821415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:31.821418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:31.821841 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:31.822122 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:31.823281 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0928 16:36:31.823612 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":241,"threads_started":1}
I20260928 16:36:31.825501 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:31.800654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37617 (local address 127.10.247.194:45541)
0928 16:36:31.801692 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:31.801765 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:31.804161 (+ 2396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:31.805251 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:31.805267 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:31.805965 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:31.806742 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:31.806766 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:31.808557 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:31.808563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:31.810513 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:31.810530 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:31.811683 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:31.811691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:31.812051 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:31.812958 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:31.812987 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:31.815849 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:31.819552 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:31.819565 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:31.819590 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:31.820458 (+ 868us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:31.820933 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:31.820938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:31.820942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:31.821275 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:31.822103 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:31.822114 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:31.822736 (+ 622us) client_negotiation.cc:769] Sending connection context
0928 16:36:31.823143 (+ 407us) client_negotiation.cc:240] Negotiation successful
0928 16:36:31.823883 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":394,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
I20260928 16:36:31.827292 11611 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37617
I20260928 16:36:31.827649 11611 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:31.828512 11611 heartbeater.cc:511] Master 127.10.247.254:37617 requested a full tablet report, sending...
I20260928 16:36:31.830313 11284 ts_manager.cc:194] Registered new tserver with Master: d10bc4e1a62c4474bf500b1d8249413b (127.10.247.194:45295)
I20260928 16:36:31.831249 11284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:45541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:36:31.851549 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37617
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:36:32.228833 11475 heartbeater.cc:503] Master 127.10.247.254:37617 was elected leader, sending a full tablet report...
W20260928 16:36:32.266716 11617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:32.267066 11617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:32.267177 11617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:32.275094 11617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:32.275244 11617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:32.275385 11617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:36:32.286306 11617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37617
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:32.287956 11617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:32.289799 11617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:32.306169 11623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:32.306639 11625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:32.308504 11622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:32.312403 11617 server_base.cc:1034] running on GCE node
I20260928 16:36:32.313272 11617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:32.314622 11617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:32.315855 11617 hybrid_clock.cc:648] HybridClock initialized: now 1790613392315788 us; error 61 us; skew 500 ppm
I20260928 16:36:32.316236 11617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:32 dist-test-slave-9fn6 krb5kdc[11238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613392, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:32.320686 11617 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:36:32.322707 11617 webserver.cc:460] Webserver started at http://127.10.247.195:41439/ using document root <none> and password file <none>
I20260928 16:36:32.324072 11617 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:32.324235 11617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:32.324646 11617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:32.328699 11617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "3e425ca3eb0347f481a65db74591a28b"
format_stamp: "Formatted at 2026-09-28 16:36:32 on dist-test-slave-9fn6"
server_key: "31b2d3e36bc7deb1f66f0ce389ae507c"
server_key_iv: "6c50bfbac4f084678517da9db2d2ad27"
server_key_version: "encryptionkey@0"
I20260928 16:36:32.329695 11617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "3e425ca3eb0347f481a65db74591a28b"
format_stamp: "Formatted at 2026-09-28 16:36:32 on dist-test-slave-9fn6"
server_key: "31b2d3e36bc7deb1f66f0ce389ae507c"
server_key_iv: "6c50bfbac4f084678517da9db2d2ad27"
server_key_version: "encryptionkey@0"
I20260928 16:36:32.336433 11617 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260928 16:36:32.340848 11633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:32.342597 11617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:36:32.342785 11617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "3e425ca3eb0347f481a65db74591a28b"
format_stamp: "Formatted at 2026-09-28 16:36:32 on dist-test-slave-9fn6"
server_key: "31b2d3e36bc7deb1f66f0ce389ae507c"
server_key_iv: "6c50bfbac4f084678517da9db2d2ad27"
server_key_version: "encryptionkey@0"
I20260928 16:36:32.343019 11617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:32.361315 11617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:32.366801 11617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:32.367172 11617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:32.368497 11617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:36:32.370302 11617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:32.370391 11617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:32.370546 11617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:32.370607 11617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:32.412010 11617 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:36475
I20260928 16:36:32.412051 11747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:36475 every 8 connection(s)
I20260928 16:36:32.413558 11617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:36:32.416203 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 11617
I20260928 16:36:32.416407 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:36:32.417040 11231 external_mini_cluster.cc:1428] Setting key 1b98f9c941edf49bdc4526c9a3847a56
Sep 28 16:36:32 dist-test-slave-9fn6 krb5kdc[11238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613392, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:32.437871 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:32.417007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:34669 (local address 127.10.247.254:37617)
0928 16:36:32.417607 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:32.417615 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:32.418945 (+ 1330us) server_negotiation.cc:407] Connection header received
0928 16:36:32.421632 (+ 2687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:32.421637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:32.421730 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:32.421852 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:32.423854 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.424887 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.426704 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.427310 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.427413 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:32.431247 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:32.431253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:32.431257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:32.431298 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:32.434738 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.435487 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.435496 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.435498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.435651 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.436062 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.436065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.436067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.436356 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:32.436477 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:32.437360 (+ 883us) server_negotiation.cc:299] Negotiation successful
0928 16:36:32.437561 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":271,"threads_started":1}
I20260928 16:36:32.439054 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:32.417739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37617 (local address 127.10.247.195:34669)
0928 16:36:32.418776 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:32.418819 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:32.421284 (+ 2465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:32.422114 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:32.422125 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:32.422808 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:32.423663 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:32.423689 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.425069 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.425074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:32.426561 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:32.426576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.427524 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.427530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:32.427759 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:32.428477 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:32.428504 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:32.430993 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:32.434877 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:32.434886 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:32.434904 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:32.435390 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:32.435732 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:32.435735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:32.435737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:32.435979 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:32.436467 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:32.436475 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:32.436979 (+ 504us) client_negotiation.cc:769] Sending connection context
0928 16:36:32.437258 (+ 279us) client_negotiation.cc:240] Negotiation successful
0928 16:36:32.437739 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":449,"threads_started":1}
I20260928 16:36:32.440697 11748 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37617
I20260928 16:36:32.441143 11748 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:32.442090 11748 heartbeater.cc:511] Master 127.10.247.254:37617 requested a full tablet report, sending...
I20260928 16:36:32.443925 11284 ts_manager.cc:194] Registered new tserver with Master: 3e425ca3eb0347f481a65db74591a28b (127.10.247.195:36475)
I20260928 16:36:32.444926 11284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:34669
I20260928 16:36:32.447677 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 16:36:32 dist-test-slave-9fn6 krb5kdc[11238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613392, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 16:36:32 dist-test-slave-9fn6 krb5kdc[11238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613392, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:36:32.503091 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:32.484909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36106 (local address 127.10.247.193:42085)
0928 16:36:32.485596 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:32.485606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:32.486124 (+ 518us) server_negotiation.cc:407] Connection header received
0928 16:36:32.486289 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:32.486297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:32.486605 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:32.486767 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:32.488705 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.489884 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.491280 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.491964 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.492065 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:32.496521 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:32.496556 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:32.496575 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:32.496667 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:32.500017 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.500878 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.500888 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.500891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.501050 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.501467 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.501470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.501472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.501928 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:32.502200 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:32.502481 (+ 281us) server_negotiation.cc:299] Negotiation successful
0928 16:36:32.502749 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":305,"threads_started":1}
W20260928 16:36:32.504741 11389 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36106
Sep 28 16:36:32 dist-test-slave-9fn6 krb5kdc[11238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 16:36:32 dist-test-slave-9fn6 krb5kdc[11238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:36:32.556761 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:32.541398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.10.247.193:42085)
0928 16:36:32.541681 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:32.541689 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:32.541887 (+ 198us) server_negotiation.cc:407] Connection header received
0928 16:36:32.542115 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:32.542119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:32.542201 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:32.542299 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:32.543802 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.544596 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.545911 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.546363 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.546484 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:32.550990 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:32.550995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:32.550998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:32.551043 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:32.553927 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.554733 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.554737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.554740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.554896 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.555398 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.555402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.555404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.555710 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:32.555910 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:32.556208 (+ 298us) server_negotiation.cc:299] Negotiation successful
0928 16:36:32.556409 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260928 16:36:32.558446 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 11345
I20260928 16:36:32.572592 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:32.559517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36134 (local address 127.10.247.193:42085)
0928 16:36:32.559718 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:32.559722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:32.560164 (+ 442us) server_negotiation.cc:407] Connection header received
0928 16:36:32.560254 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:32.560258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:32.560319 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:32.560396 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:32.561861 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.562612 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.564184 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.564678 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.564769 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:32.567332 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:32.567338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:32.567341 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:32.567373 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:32.570147 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.570809 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.570813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.570816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.570952 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.571326 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.571330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.571332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.571608 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:32.571796 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:32.572095 (+ 299us) server_negotiation.cc:299] Negotiation successful
0928 16:36:32.572267 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 16:36:32.573441 11470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:32.664547 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 11345
I20260928 16:36:32.681093 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 11481
Sep 28 16:36:32 dist-test-slave-9fn6 krb5kdc[11238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:36:32.698939 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:32.682231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38186 (local address 127.10.247.194:45295)
0928 16:36:32.682941 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:32.682950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:32.682985 (+ 35us) server_negotiation.cc:407] Connection header received
0928 16:36:32.683073 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:32.683081 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:32.683374 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:32.683563 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:32.684997 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.686190 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.687567 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.688221 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.688328 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:32.692697 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:32.692712 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:32.692731 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:32.692774 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:32.695725 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.696454 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.696462 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.696465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.696887 (+ 422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.697305 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.697309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.697311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.697756 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:32.698023 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:32.698309 (+ 286us) server_negotiation.cc:299] Negotiation successful
0928 16:36:32.698577 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":338,"threads_started":1}
I20260928 16:36:32.700646 11606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:32.789528 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 11481
I20260928 16:36:32.806277 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 11617
Sep 28 16:36:32 dist-test-slave-9fn6 krb5kdc[11238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:36:32.824497 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:32.807409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.10.247.195:36475)
0928 16:36:32.808086 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:32.808095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:32.808133 (+ 38us) server_negotiation.cc:407] Connection header received
0928 16:36:32.808234 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:32.808242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:32.808575 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:32.808774 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:32.810140 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.811279 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.812797 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:32.813447 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:32.813559 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:32.818114 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:32.818138 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:32.818184 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:32.818238 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:32.821485 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.822261 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.822270 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.822272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.822420 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:32.822782 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:32.822785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:32.822787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:32.823219 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:32.823493 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:32.823799 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 16:36:32.824150 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":310,"threads_started":1}
I20260928 16:36:32.826112 11743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:32.915274 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 11617
I20260928 16:36:32.931775 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 11251
I20260928 16:36:32.933346 11314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:33.060364 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 11251
2026-09-28T16:36:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4508 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:36:33 dist-test-slave-9fn6 krb5kdc[11770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:36:33 dist-test-slave-9fn6 krb5kdc[11770](info): set up 2 sockets
Sep 28 16:36:33 dist-test-slave-9fn6 krb5kdc[11770](info): commencing operation
krb5kdc: starting...
W20260928 16:36:34.085059 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:36:34 dist-test-slave-9fn6 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:36:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:36:34.300024 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42957
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:42957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:34.679333 11783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:34.679716 11783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:34.679857 11783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:34.687752 11783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:36:34.687903 11783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:34.687952 11783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:34.688040 11783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:36:34.688102 11783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:36:34.699254 11783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:42957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:34.701078 11783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:34.702706 11783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:34.716099 11789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:34.716027 11791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:34.716058 11788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:34.717051 11783 server_base.cc:1034] running on GCE node
I20260928 16:36:34.718024 11783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:34.719789 11783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:34.721001 11783 hybrid_clock.cc:648] HybridClock initialized: now 1790613394720954 us; error 33 us; skew 500 ppm
I20260928 16:36:34.721460 11783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:34 dist-test-slave-9fn6 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613394, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:34.731781 11783 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:36:34.734005 11783 webserver.cc:460] Webserver started at http://127.10.247.254:35197/ using document root <none> and password file <none>
I20260928 16:36:34.734949 11783 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:34.735095 11783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:34.735486 11783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:34.739365 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "b6fbdaf8b0944760a5d8bfe3428abf37"
format_stamp: "Formatted at 2026-09-28 16:36:34 on dist-test-slave-9fn6"
server_key: "7d50e815c33e858ee436f1fcefe82747"
server_key_iv: "812ebea2fe2ccc9d32ab9de5a88d6bb1"
server_key_version: "encryptionkey@0"
I20260928 16:36:34.740268 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "b6fbdaf8b0944760a5d8bfe3428abf37"
format_stamp: "Formatted at 2026-09-28 16:36:34 on dist-test-slave-9fn6"
server_key: "7d50e815c33e858ee436f1fcefe82747"
server_key_iv: "812ebea2fe2ccc9d32ab9de5a88d6bb1"
server_key_version: "encryptionkey@0"
I20260928 16:36:34.746963 11783 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:36:34.751463 11798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:34.753319 11783 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:36:34.753549 11783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "b6fbdaf8b0944760a5d8bfe3428abf37"
format_stamp: "Formatted at 2026-09-28 16:36:34 on dist-test-slave-9fn6"
server_key: "7d50e815c33e858ee436f1fcefe82747"
server_key_iv: "812ebea2fe2ccc9d32ab9de5a88d6bb1"
server_key_version: "encryptionkey@0"
I20260928 16:36:34.753808 11783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:34.776492 11783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:34.781579 11783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:34.781960 11783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:34.802635 11783 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:42957
I20260928 16:36:34.802656 11849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:42957 every 8 connection(s)
I20260928 16:36:34.804318 11783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:36:34.808893 11850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:34.813016 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 11783
I20260928 16:36:34.813316 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:36:34.814186 11231 external_mini_cluster.cc:1428] Setting key 577ac23fe914afa4ce1cdbd6c5c20d6d
I20260928 16:36:34.824199 11850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37: Bootstrap starting.
I20260928 16:36:34.829034 11850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:34.830560 11850 log.cc:824] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37: Log is configured to *not* fsync() on all Append() calls
Sep 28 16:36:34 dist-test-slave-9fn6 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:34.834252 11850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37: No bootstrap required, opened a new log
I20260928 16:36:34.840349 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:34.816744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51678 (local address 127.10.247.254:42957)
0928 16:36:34.817621 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:34.817636 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:34.817684 (+ 48us) server_negotiation.cc:407] Connection header received
0928 16:36:34.819468 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:34.819498 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:34.820010 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:34.820391 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:34.822468 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:34.823965 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:34.825840 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:34.826676 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:34.826840 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:34.831927 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:34.831937 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:34.831956 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:34.832000 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:34.835302 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:34.836098 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:34.836113 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:34.836121 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:34.836326 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:34.836766 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:34.836771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:34.836773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:34.837482 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:34.837841 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:34.838461 (+ 620us) server_negotiation.cc:299] Negotiation successful
0928 16:36:34.839003 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":389,"threads_started":1}
I20260928 16:36:34.843068 11850 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fbdaf8b0944760a5d8bfe3428abf37" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42957 } }
I20260928 16:36:34.843518 11850 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:34.843629 11850 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6fbdaf8b0944760a5d8bfe3428abf37, State: Initialized, Role: FOLLOWER
I20260928 16:36:34.844241 11850 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fbdaf8b0944760a5d8bfe3428abf37" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42957 } }
I20260928 16:36:34.844493 11850 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:36:34.844652 11850 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:36:34.844818 11850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:34.847270 11850 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fbdaf8b0944760a5d8bfe3428abf37" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42957 } }
I20260928 16:36:34.847873 11850 leader_election.cc:302] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6fbdaf8b0944760a5d8bfe3428abf37; no voters:
I20260928 16:36:34.848668 11850 leader_election.cc:288] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:36:34.848807 11857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:34.849493 11857 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [term 1 LEADER]: Becoming Leader. State: Replica: b6fbdaf8b0944760a5d8bfe3428abf37, State: Running, Role: LEADER
I20260928 16:36:34.850078 11857 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fbdaf8b0944760a5d8bfe3428abf37" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42957 } }
I20260928 16:36:34.851044 11850 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:36:34.853045 11859 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6fbdaf8b0944760a5d8bfe3428abf37. Latest consensus state: current_term: 1 leader_uuid: "b6fbdaf8b0944760a5d8bfe3428abf37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fbdaf8b0944760a5d8bfe3428abf37" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42957 } } }
I20260928 16:36:34.853518 11859 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:34.854475 11866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:36:34.854275 11858 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6fbdaf8b0944760a5d8bfe3428abf37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fbdaf8b0944760a5d8bfe3428abf37" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42957 } } }
I20260928 16:36:34.855243 11858 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37 [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:34.860476 11866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:36:34.870757 11866 catalog_manager.cc:1347] Generated new cluster ID: 816459db8f37499b9a4dd10f82e3824c
I20260928 16:36:34.870909 11866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:36:34.904580 11866 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:36:34.906950 11866 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:36:34.919967 11866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6fbdaf8b0944760a5d8bfe3428abf37: Generated new TSK 0
I20260928 16:36:34.921514 11866 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:36:34.977061 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42957
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:35.371688 11878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:35.372102 11878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:35.372262 11878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:35.380287 11878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:35.380468 11878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:35.380668 11878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:36:35.391930 11878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42957
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:35.393757 11878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:35.395565 11878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:35.411065 11883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:35.410966 11886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:35.411202 11884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:35.412043 11878 server_base.cc:1034] running on GCE node
I20260928 16:36:35.412875 11878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:35.414026 11878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:35.415244 11878 hybrid_clock.cc:648] HybridClock initialized: now 1790613395415168 us; error 64 us; skew 500 ppm
I20260928 16:36:35.415619 11878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:35 dist-test-slave-9fn6 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613395, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:35.419430 11878 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:36:35.421393 11878 webserver.cc:460] Webserver started at http://127.10.247.193:45261/ using document root <none> and password file <none>
I20260928 16:36:35.422320 11878 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:35.422462 11878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:35.422844 11878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:35.426616 11878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "ae3946dc21364fc697d4882b164698b2"
format_stamp: "Formatted at 2026-09-28 16:36:35 on dist-test-slave-9fn6"
server_key: "da34d941d61caa46303bf9668fac37cd"
server_key_iv: "db6ce8286ee7e17209f8f89d1b7928b5"
server_key_version: "encryptionkey@0"
I20260928 16:36:35.427552 11878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "ae3946dc21364fc697d4882b164698b2"
format_stamp: "Formatted at 2026-09-28 16:36:35 on dist-test-slave-9fn6"
server_key: "da34d941d61caa46303bf9668fac37cd"
server_key_iv: "db6ce8286ee7e17209f8f89d1b7928b5"
server_key_version: "encryptionkey@0"
I20260928 16:36:35.434134 11878 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260928 16:36:35.438725 11893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:35.440475 11878 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260928 16:36:35.440748 11878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "ae3946dc21364fc697d4882b164698b2"
format_stamp: "Formatted at 2026-09-28 16:36:35 on dist-test-slave-9fn6"
server_key: "da34d941d61caa46303bf9668fac37cd"
server_key_iv: "db6ce8286ee7e17209f8f89d1b7928b5"
server_key_version: "encryptionkey@0"
I20260928 16:36:35.441004 11878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:35.466174 11878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:35.471345 11878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:35.471664 11878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:35.472986 11878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:36:35.474676 11878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:35.474769 11878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:35.474906 11878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:35.474975 11878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:35.509398 11878 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:35485
I20260928 16:36:35.509449 12007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:35485 every 8 connection(s)
I20260928 16:36:35.510998 11878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:36:35.513598 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 11878
I20260928 16:36:35.513833 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:36:35.514756 11231 external_mini_cluster.cc:1428] Setting key f01ef36bfc36806c1a11d34ca5861de7
Sep 28 16:36:35 dist-test-slave-9fn6 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613395, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
I20260928 16:36:35.535575 12010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:35.514183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:51115 (local address 127.10.247.254:42957)
0928 16:36:35.514778 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:35.514786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:35.515995 (+ 1209us) server_negotiation.cc:407] Connection header received
0928 16:36:35.518284 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:35.518291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:35.518390 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:35.518528 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:35.520281 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:35.521343 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:35.523130 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:35.523585 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:35.523712 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:35.527973 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:35.527980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:35.527985 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:35.528032 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:35.531589 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:35.532588 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:35.532595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:35.532599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:35.532796 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:35.533322 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:35.533328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:35.533332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:35.533756 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:35.533937 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:35.534871 (+ 934us) server_negotiation.cc:299] Negotiation successful
0928 16:36:35.535173 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":273,"threads_started":1}
I20260928 16:36:35.537245 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:35.514825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42957 (local address 127.10.247.193:51115)
0928 16:36:35.515799 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:35.515865 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:35.517976 (+ 2111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:35.518755 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:35.518765 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:35.519386 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:35.520091 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:35.520111 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:35.521548 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:35.521553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:35.522963 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:35.522976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:35.523850 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:35.523857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:35.524153 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:35.524944 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:35.524973 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:35.527666 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:35.531808 (+ 4142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:35.531818 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:35.531838 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:35.532432 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:35.532932 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:35.532936 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:35.532939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:35.533193 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:35.533922 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:35.533935 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:35.534397 (+ 462us) client_negotiation.cc:769] Sending connection context
0928 16:36:35.534740 (+ 343us) client_negotiation.cc:240] Negotiation successful
0928 16:36:35.535527 (+ 787us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":350,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
I20260928 16:36:35.539324 12008 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42957
I20260928 16:36:35.539768 12008 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:35.541109 12008 heartbeater.cc:511] Master 127.10.247.254:42957 requested a full tablet report, sending...
I20260928 16:36:35.544258 11815 ts_manager.cc:194] Registered new tserver with Master: ae3946dc21364fc697d4882b164698b2 (127.10.247.193:35485)
I20260928 16:36:35.546823 11815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:51115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:36:35.566118 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42957
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:35.948666 12014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:35.948997 12014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:35.949157 12014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:35.957012 12014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:35.957134 12014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:35.957296 12014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:36:35.968703 12014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42957
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:35.970445 12014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:35.972056 12014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:35.987879 12020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:35.987838 12022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:35.987905 12019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:35.994215 12014 server_base.cc:1034] running on GCE node
I20260928 16:36:35.995093 12014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:35.996331 12014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:35.997593 12014 hybrid_clock.cc:648] HybridClock initialized: now 1790613395997484 us; error 81 us; skew 500 ppm
I20260928 16:36:35.997967 12014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:35 dist-test-slave-9fn6 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613395, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:36.001664 12014 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:36:36.003598 12014 webserver.cc:460] Webserver started at http://127.10.247.194:34653/ using document root <none> and password file <none>
I20260928 16:36:36.004472 12014 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:36.004594 12014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:36.004992 12014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:36.008574 12014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "57f7bf387e49444694320787bda3160d"
format_stamp: "Formatted at 2026-09-28 16:36:36 on dist-test-slave-9fn6"
server_key: "de3c4b6d5397fc37af018a035b71143c"
server_key_iv: "3b2b90f69ea44e02db25b98842720427"
server_key_version: "encryptionkey@0"
I20260928 16:36:36.009502 12014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "57f7bf387e49444694320787bda3160d"
format_stamp: "Formatted at 2026-09-28 16:36:36 on dist-test-slave-9fn6"
server_key: "de3c4b6d5397fc37af018a035b71143c"
server_key_iv: "3b2b90f69ea44e02db25b98842720427"
server_key_version: "encryptionkey@0"
I20260928 16:36:36.015605 12014 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:36:36.019663 12029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:36.021255 12014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:36:36.021420 12014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "57f7bf387e49444694320787bda3160d"
format_stamp: "Formatted at 2026-09-28 16:36:36 on dist-test-slave-9fn6"
server_key: "de3c4b6d5397fc37af018a035b71143c"
server_key_iv: "3b2b90f69ea44e02db25b98842720427"
server_key_version: "encryptionkey@0"
I20260928 16:36:36.021631 12014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:36.047214 12014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:36.052644 12014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:36.052983 12014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:36.054147 12014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:36:36.055727 12014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:36.055812 12014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:36.055941 12014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:36.055989 12014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:36.089661 12014 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:33125
I20260928 16:36:36.089709 12143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:33125 every 8 connection(s)
I20260928 16:36:36.091199 12014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:36:36.099586 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 12014
I20260928 16:36:36.099807 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:36:36.100482 11231 external_mini_cluster.cc:1428] Setting key f416614779bdd61d852ba029715b3e16
Sep 28 16:36:36 dist-test-slave-9fn6 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613395, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:36.115034 12146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.094219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:51213 (local address 127.10.247.254:42957)
0928 16:36:36.094744 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.094750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.095710 (+ 960us) server_negotiation.cc:407] Connection header received
0928 16:36:36.097903 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.097910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.098009 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.098121 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:36.100010 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.101161 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.103402 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.104075 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.104230 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.108314 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:36.108322 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:36.108328 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:36.108372 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:36.111260 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.112256 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.112261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.112263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.112412 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.112920 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.112925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.112928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.113369 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:36.113503 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.114486 (+ 983us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.114712 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":272,"threads_started":1}
I20260928 16:36:36.116752 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.094728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42957 (local address 127.10.247.194:51213)
0928 16:36:36.095522 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:36.095567 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:36.097575 (+ 2008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:36.098416 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:36.098429 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:36.099053 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:36.099779 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.099807 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.101378 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.101385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.103248 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.103264 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.104327 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.104333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.104585 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.105461 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:36.105491 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:36.108049 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:36.111422 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:36.111433 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:36.111457 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:36.112120 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:36.112509 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:36.112514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:36.112517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:36.112815 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:36.113492 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:36.113501 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:36.114030 (+ 529us) client_negotiation.cc:769] Sending connection context
0928 16:36:36.114385 (+ 355us) client_negotiation.cc:240] Negotiation successful
0928 16:36:36.115062 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":295,"threads_started":1}
I20260928 16:36:36.118649 12144 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42957
I20260928 16:36:36.119015 12144 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:36.119930 12144 heartbeater.cc:511] Master 127.10.247.254:42957 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
I20260928 16:36:36.121887 11815 ts_manager.cc:194] Registered new tserver with Master: 57f7bf387e49444694320787bda3160d (127.10.247.194:33125)
I20260928 16:36:36.122871 11815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:51213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:36:36.148806 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42957
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:36.518805 12150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:36.519172 12150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:36.519330 12150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:36.527453 12150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:36.527612 12150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:36.527784 12150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:36:36.539101 12150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42957
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:36.540870 12150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:36.542526 12150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 16:36:36.551322 12008 heartbeater.cc:503] Master 127.10.247.254:42957 was elected leader, sending a full tablet report...
W20260928 16:36:36.557746 12155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:36.557786 12156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:36.557708 12158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:36.558559 12150 server_base.cc:1034] running on GCE node
I20260928 16:36:36.559324 12150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:36.560410 12150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:36.561631 12150 hybrid_clock.cc:648] HybridClock initialized: now 1790613396561554 us; error 61 us; skew 500 ppm
I20260928 16:36:36.561985 12150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:36 dist-test-slave-9fn6 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:36.565444 12150 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:36:36.567142 12150 webserver.cc:460] Webserver started at http://127.10.247.195:35441/ using document root <none> and password file <none>
I20260928 16:36:36.567986 12150 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:36.568120 12150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:36.568466 12150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:36.572009 12150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "bcd950a563da4da192b25373bbd7048d"
format_stamp: "Formatted at 2026-09-28 16:36:36 on dist-test-slave-9fn6"
server_key: "9ac02d68dfcca8a8a4948862246ebf86"
server_key_iv: "cbee70e404bae77e0c170a776418e154"
server_key_version: "encryptionkey@0"
I20260928 16:36:36.572981 12150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "bcd950a563da4da192b25373bbd7048d"
format_stamp: "Formatted at 2026-09-28 16:36:36 on dist-test-slave-9fn6"
server_key: "9ac02d68dfcca8a8a4948862246ebf86"
server_key_iv: "cbee70e404bae77e0c170a776418e154"
server_key_version: "encryptionkey@0"
I20260928 16:36:36.579382 12150 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260928 16:36:36.583861 12165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:36.585537 12150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:36:36.585728 12150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "bcd950a563da4da192b25373bbd7048d"
format_stamp: "Formatted at 2026-09-28 16:36:36 on dist-test-slave-9fn6"
server_key: "9ac02d68dfcca8a8a4948862246ebf86"
server_key_iv: "cbee70e404bae77e0c170a776418e154"
server_key_version: "encryptionkey@0"
I20260928 16:36:36.585959 12150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:36.604833 12150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:36.610067 12150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:36.610381 12150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:36.611567 12150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:36:36.613286 12150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:36.613368 12150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:36.613497 12150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:36.613548 12150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:36.650373 12150 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:36617
I20260928 16:36:36.650419 12277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:36617 every 8 connection(s)
I20260928 16:36:36.652052 12150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:36:36.661294 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 12150
I20260928 16:36:36.661553 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:36:36.662045 11231 external_mini_cluster.cc:1428] Setting key b0ea0742f5e682828ebea2480e4495ac
Sep 28 16:36:36 dist-test-slave-9fn6 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:36.674685 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.655348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:56775 (local address 127.10.247.254:42957)
0928 16:36:36.655889 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.655894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.657091 (+ 1197us) server_negotiation.cc:407] Connection header received
0928 16:36:36.659274 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.659281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.659395 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.659507 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:36.661587 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.662552 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.664360 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.664852 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.664956 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.669220 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:36.669226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:36.669230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:36.669271 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:36.671883 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.672541 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.672544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.672547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.672697 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.673061 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.673064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.673066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.673354 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:36.673474 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.674174 (+ 700us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.674361 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":288,"threads_started":1}
I20260928 16:36:36.676000 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.656034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42957 (local address 127.10.247.195:56775)
0928 16:36:36.656912 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:36.656964 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:36.658955 (+ 1991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:36.659807 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:36.659816 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:36.660443 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:36.661372 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.661401 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.662725 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.662729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.664223 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.664237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.665093 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.665101 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.665343 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.666114 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:36.666145 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:36.668984 (+ 2839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:36.671990 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:36.671998 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:36.672012 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:36.672444 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:36.672786 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:36.672790 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:36.672792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:36.672980 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:36.673443 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:36.673449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:36.673835 (+ 386us) client_negotiation.cc:769] Sending connection context
0928 16:36:36.674073 (+ 238us) client_negotiation.cc:240] Negotiation successful
0928 16:36:36.674521 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":320,"threads_started":1}
I20260928 16:36:36.677517 12278 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42957
I20260928 16:36:36.677851 12278 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:36.678748 12278 heartbeater.cc:511] Master 127.10.247.254:42957 requested a full tablet report, sending...
I20260928 16:36:36.680415 11814 ts_manager.cc:194] Registered new tserver with Master: bcd950a563da4da192b25373bbd7048d (127.10.247.195:36617)
I20260928 16:36:36.681531 11814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:56775
I20260928 16:36:36.691048 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:36:36.712001 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.698774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.10.247.254:42957)
0928 16:36:36.698963 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.698969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.699551 (+ 582us) server_negotiation.cc:407] Connection header received
0928 16:36:36.699751 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.699755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.699838 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.699919 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:36.701704 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.702535 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.703816 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.704265 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.704366 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.706776 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:36.706782 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:36.706786 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:36.706826 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:36.709471 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.710178 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.710182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.710185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.710340 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.710650 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.710653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.710655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.711038 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:36.711254 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.711502 (+ 248us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.711708 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:36:36.720434 11814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:36:36.728808 11814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:36:36.772236 12296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.745732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:35485 (local address 127.0.0.1:57432)
0928 16:36:36.762566 (+ 16834us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:36.762588 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:36.762745 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:36.764842 (+ 2097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:36.764851 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:36.764886 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:36.765335 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.765348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.767163 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.767167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.769457 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.769474 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.770979 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.770985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.771316 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.771374 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 16:36:36.771492 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 16:36:36.771751 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16671,"mutex_wait_us":49,"thread_start_us":16509,"threads_started":1}
I20260928 16:36:36.772296 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.745199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.10.247.193:35485)
0928 16:36:36.763493 (+ 18294us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.763506 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.763549 (+ 43us) server_negotiation.cc:407] Connection header received
0928 16:36:36.763716 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.763727 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.764119 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.764393 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:36.765568 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.767010 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.769658 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.770746 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.770956 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.771198 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.771596 (+ 398us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.771783 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18062,"thread_start_us":17866,"threads_started":1}
I20260928 16:36:36.775163 12292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.744178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:36617 (local address 127.0.0.1:40292)
0928 16:36:36.757414 (+ 13236us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:36.757482 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:36.757819 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:36.766034 (+ 8215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:36.766039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:36.766067 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:36.766898 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.766913 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.769057 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.769063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.771812 (+ 2749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.771827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.774209 (+ 2382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.774216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.774398 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.774436 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 16:36:36.774522 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 16:36:36.774664 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12798,"mutex_wait_us":150,"thread_start_us":16221,"threads_started":1}
I20260928 16:36:36.775298 12294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.745016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.10.247.194:33125)
0928 16:36:36.764085 (+ 19069us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.764095 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.764131 (+ 36us) server_negotiation.cc:407] Connection header received
0928 16:36:36.764241 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.764248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.765539 (+ 1291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.765729 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:36.766447 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.768219 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.772010 (+ 3791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.773915 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.774069 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.774235 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.774721 (+ 486us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.774913 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18867,"thread_start_us":18733,"threads_started":1}
I20260928 16:36:36.775906 12293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.744869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40292 (local address 127.10.247.195:36617)
0928 16:36:36.765094 (+ 20225us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.765103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.765139 (+ 36us) server_negotiation.cc:407] Connection header received
0928 16:36:36.765243 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.765250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.765473 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.765720 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:36.767208 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.768875 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.771997 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.773690 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.773882 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.774068 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.775463 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.775609 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20034,"thread_start_us":19858,"threads_started":1}
I20260928 16:36:36.775187 12291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.744369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:33125 (local address 127.0.0.1:33052)
0928 16:36:36.760127 (+ 15758us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:36.760152 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:36.760310 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:36.765747 (+ 5437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:36.765752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:36.765780 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:36.766227 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.766241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.768392 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.768398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.771813 (+ 3415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.771827 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.774369 (+ 2542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.774375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.774531 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.774571 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 16:36:36.774643 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 16:36:36.774755 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15588,"mutex_wait_us":253,"thread_start_us":12384,"threads_started":1}
I20260928 16:36:36.777436 11942 tablet_service.cc:1467] Processing CreateTablet for tablet 698e26666be14ab2b0d5d61352ab20c1 (DEFAULT_TABLE table=test-table [id=61fae6189d1341cf834801d670a8451f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:36.779361 11942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 698e26666be14ab2b0d5d61352ab20c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:36.779969 12078 tablet_service.cc:1467] Processing CreateTablet for tablet 698e26666be14ab2b0d5d61352ab20c1 (DEFAULT_TABLE table=test-table [id=61fae6189d1341cf834801d670a8451f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:36.779968 12213 tablet_service.cc:1467] Processing CreateTablet for tablet 698e26666be14ab2b0d5d61352ab20c1 (DEFAULT_TABLE table=test-table [id=61fae6189d1341cf834801d670a8451f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:36.781499 12078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 698e26666be14ab2b0d5d61352ab20c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:36.781519 12213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 698e26666be14ab2b0d5d61352ab20c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:36.795475 12297 tablet_bootstrap.cc:492] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2: Bootstrap starting.
I20260928 16:36:36.799404 12297 tablet_bootstrap.cc:654] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:36.800479 12299 tablet_bootstrap.cc:492] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d: Bootstrap starting.
I20260928 16:36:36.801564 12297 log.cc:824] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:36.808352 12299 tablet_bootstrap.cc:654] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:36.809599 12298 tablet_bootstrap.cc:492] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d: Bootstrap starting.
I20260928 16:36:36.810056 12299 log.cc:824] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:36.814541 12298 tablet_bootstrap.cc:654] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:36.816381 12298 log.cc:824] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:36.823086 12297 tablet_bootstrap.cc:492] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2: No bootstrap required, opened a new log
I20260928 16:36:36.823092 12299 tablet_bootstrap.cc:492] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d: No bootstrap required, opened a new log
I20260928 16:36:36.823453 12297 ts_tablet_manager.cc:1397] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.016s
I20260928 16:36:36.823482 12299 ts_tablet_manager.cc:1397] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d: Time spent bootstrapping tablet: real 0.023s user 0.015s sys 0.000s
I20260928 16:36:36.831071 12298 tablet_bootstrap.cc:492] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d: No bootstrap required, opened a new log
I20260928 16:36:36.831444 12298 ts_tablet_manager.cc:1397] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.004s
I20260928 16:36:36.833333 12297 raft_consensus.cc:348] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } }
I20260928 16:36:36.833858 12297 raft_consensus.cc:374] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:36.833964 12297 raft_consensus.cc:729] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae3946dc21364fc697d4882b164698b2, State: Initialized, Role: FOLLOWER
I20260928 16:36:36.834678 12297 consensus_queue.cc:260] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } }
I20260928 16:36:36.836823 12297 ts_tablet_manager.cc:1428] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2: Time spent starting tablet: real 0.013s user 0.015s sys 0.001s
I20260928 16:36:36.837183 12299 raft_consensus.cc:348] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } }
I20260928 16:36:36.837723 12299 raft_consensus.cc:374] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:36.837841 12299 raft_consensus.cc:729] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd950a563da4da192b25373bbd7048d, State: Initialized, Role: FOLLOWER
I20260928 16:36:36.838742 12299 consensus_queue.cc:260] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } }
I20260928 16:36:36.840114 12278 heartbeater.cc:503] Master 127.10.247.254:42957 was elected leader, sending a full tablet report...
I20260928 16:36:36.842519 12299 ts_tablet_manager.cc:1428] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d: Time spent starting tablet: real 0.019s user 0.017s sys 0.000s
I20260928 16:36:36.842864 12298 raft_consensus.cc:348] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } }
I20260928 16:36:36.843533 12298 raft_consensus.cc:374] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:36.843665 12298 raft_consensus.cc:729] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f7bf387e49444694320787bda3160d, State: Initialized, Role: FOLLOWER
I20260928 16:36:36.844462 12298 consensus_queue.cc:260] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } }
I20260928 16:36:36.845783 12144 heartbeater.cc:503] Master 127.10.247.254:42957 was elected leader, sending a full tablet report...
I20260928 16:36:36.846616 12298 ts_tablet_manager.cc:1428] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260928 16:36:36.856968 12304 raft_consensus.cc:482] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:36:36.857349 12304 raft_consensus.cc:504] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } }
I20260928 16:36:36.859633 12304 leader_election.cc:288] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57f7bf387e49444694320787bda3160d (127.10.247.194:33125), ae3946dc21364fc697d4882b164698b2 (127.10.247.193:35485)
I20260928 16:36:36.866161 12294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.860241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:51091 (local address 127.10.247.194:33125)
0928 16:36:36.860454 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.860459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.860483 (+ 24us) server_negotiation.cc:407] Connection header received
0928 16:36:36.860581 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.860587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.860715 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.860830 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:36.861263 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.862262 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.864329 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.865230 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.865379 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.865467 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.865732 (+ 265us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.865875 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 16:36:36.866166 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.859916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:33125 (local address 127.10.247.195:51091)
0928 16:36:36.860130 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:36.860154 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:36.860308 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:36.860862 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:36.860865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:36.860899 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:36.861106 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.861116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.862446 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.862452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.864157 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.864173 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.865379 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.865385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.865575 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.865602 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 16:36:36.865684 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 16:36:36.865825 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":24}
I20260928 16:36:36.866823 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.860134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:37079 (local address 127.10.247.193:35485)
0928 16:36:36.860368 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.860375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.861097 (+ 722us) server_negotiation.cc:407] Connection header received
0928 16:36:36.861387 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.861392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.861494 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.861587 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:36.862110 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.863008 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.865016 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.865886 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.866042 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.866119 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.866484 (+ 365us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.866624 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260928 16:36:36.867468 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.859916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:35485 (local address 127.10.247.195:37079)
0928 16:36:36.860970 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:36.860987 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:36.861122 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:36.861611 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:36.861614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:36.861635 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:36.861962 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.861971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.863162 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.863166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.864858 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:36.864871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.866096 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.866101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:36.866294 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.866332 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 16:36:36.867009 (+ 677us) client_negotiation.cc:240] Negotiation successful
0928 16:36:36.867129 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"mutex_wait_us":243,"thread_start_us":372,"threads_started":1}
I20260928 16:36:36.867435 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698e26666be14ab2b0d5d61352ab20c1" candidate_uuid: "bcd950a563da4da192b25373bbd7048d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f7bf387e49444694320787bda3160d" is_pre_election: true
I20260928 16:36:36.868149 12098 raft_consensus.cc:2454] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcd950a563da4da192b25373bbd7048d in term 0.
I20260928 16:36:36.868467 11962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698e26666be14ab2b0d5d61352ab20c1" candidate_uuid: "bcd950a563da4da192b25373bbd7048d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae3946dc21364fc697d4882b164698b2" is_pre_election: true
I20260928 16:36:36.868959 12167 leader_election.cc:302] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57f7bf387e49444694320787bda3160d, bcd950a563da4da192b25373bbd7048d; no voters:
I20260928 16:36:36.869037 11962 raft_consensus.cc:2454] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcd950a563da4da192b25373bbd7048d in term 0.
I20260928 16:36:36.869400 12304 raft_consensus.cc:2781] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:36:36.869560 12304 raft_consensus.cc:482] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:36:36.869676 12304 raft_consensus.cc:3037] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:36.872174 12304 raft_consensus.cc:504] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } }
I20260928 16:36:36.872864 12304 leader_election.cc:288] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [CANDIDATE]: Term 1 election: Requested vote from peers 57f7bf387e49444694320787bda3160d (127.10.247.194:33125), ae3946dc21364fc697d4882b164698b2 (127.10.247.193:35485)
I20260928 16:36:36.873333 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698e26666be14ab2b0d5d61352ab20c1" candidate_uuid: "bcd950a563da4da192b25373bbd7048d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f7bf387e49444694320787bda3160d"
I20260928 16:36:36.873378 11962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698e26666be14ab2b0d5d61352ab20c1" candidate_uuid: "bcd950a563da4da192b25373bbd7048d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae3946dc21364fc697d4882b164698b2"
I20260928 16:36:36.873571 12098 raft_consensus.cc:3037] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:36.873579 11962 raft_consensus.cc:3037] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:36.876091 12098 raft_consensus.cc:2454] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcd950a563da4da192b25373bbd7048d in term 1.
I20260928 16:36:36.876214 11962 raft_consensus.cc:2454] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcd950a563da4da192b25373bbd7048d in term 1.
I20260928 16:36:36.876524 12167 leader_election.cc:302] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57f7bf387e49444694320787bda3160d, bcd950a563da4da192b25373bbd7048d; no voters:
I20260928 16:36:36.876868 12304 raft_consensus.cc:2781] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:36.877378 12304 raft_consensus.cc:686] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [term 1 LEADER]: Becoming Leader. State: Replica: bcd950a563da4da192b25373bbd7048d, State: Running, Role: LEADER
I20260928 16:36:36.878001 12304 consensus_queue.cc:237] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } }
I20260928 16:36:36.883473 11814 catalog_manager.cc:5495] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d reported cstate change: term changed from 0 to 1, leader changed from <none> to bcd950a563da4da192b25373bbd7048d (127.10.247.195). New cstate: current_term: 1 leader_uuid: "bcd950a563da4da192b25373bbd7048d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd950a563da4da192b25373bbd7048d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57f7bf387e49444694320787bda3160d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae3946dc21364fc697d4882b164698b2" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35485 } health_report { overall_health: UNKNOWN } } }
Sep 28 16:36:36 dist-test-slave-9fn6 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
W20260928 16:36:36.904294 12279 tablet.cc:2369] T 698e26666be14ab2b0d5d61352ab20c1 P bcd950a563da4da192b25373bbd7048d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:36:36.908514 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.891129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.10.247.193:35485)
0928 16:36:36.891327 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.891335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.891790 (+ 455us) server_negotiation.cc:407] Connection header received
0928 16:36:36.892003 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.892007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.892089 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.892193 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:36.894280 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.895141 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.896815 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.897323 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.897450 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.901962 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:36.901984 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:36.902005 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:36.902049 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:36.905086 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.906065 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.906075 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.906077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.906246 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.906643 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.906646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.906649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.907104 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:36.907438 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.907760 (+ 322us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.908021 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Sep 28 16:36:36 dist-test-slave-9fn6 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613396, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 16:36:36 dist-test-slave-9fn6 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613396, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:36:36.963119 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.947280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57454 (local address 127.10.247.193:35485)
0928 16:36:36.947539 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.947547 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.947937 (+ 390us) server_negotiation.cc:407] Connection header received
0928 16:36:36.948135 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:36.948139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:36.948231 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:36.948322 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:36.950054 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.950879 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.952445 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:36.953000 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:36.953105 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:36.957761 (+ 4656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:36.957766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:36.957770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:36.957814 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:36.960683 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.961283 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.961286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.961289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.961451 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:36.961851 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:36.961854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:36.961856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:36.962149 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:36.962392 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:36.962613 (+ 221us) server_negotiation.cc:299] Negotiation successful
0928 16:36:36.962814 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Sep 28 16:36:36 dist-test-slave-9fn6 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 16:36:37 dist-test-slave-9fn6 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
W20260928 16:36:37.013814 12009 tablet.cc:2369] T 698e26666be14ab2b0d5d61352ab20c1 P ae3946dc21364fc697d4882b164698b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:36:37.015914 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:36.999461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.10.247.193:35485)
0928 16:36:36.999726 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:36.999735 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:36.999881 (+ 146us) server_negotiation.cc:407] Connection header received
0928 16:36:37.000125 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:37.000129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:37.000211 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:37.000306 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:37.002074 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:37.002885 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:37.004318 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:37.004849 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:37.004948 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:37.009421 (+ 4473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:37.009427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:37.009431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:37.009486 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:37.012842 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:37.013642 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:37.013649 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:37.013651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:37.013822 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:37.014250 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:37.014255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:37.014257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:37.014659 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:37.014923 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:37.015245 (+ 322us) server_negotiation.cc:299] Negotiation successful
0928 16:36:37.015521 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260928 16:36:37.018419 11922 tablet_service.cc:2908] Rejecting scan request for tablet 698e26666be14ab2b0d5d61352ab20c1: Uninitialized: clean time has not yet been initialized
I20260928 16:36:37.021970 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 11878
I20260928 16:36:37.036520 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:37.023205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57480 (local address 127.10.247.193:35485)
0928 16:36:37.023445 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:37.023453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:37.023743 (+ 290us) server_negotiation.cc:407] Connection header received
0928 16:36:37.023908 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:37.023913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:37.024004 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:37.024100 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:37.025450 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:37.026273 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:37.027638 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:37.028113 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:37.028213 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:37.030654 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:37.030660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:37.030663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:37.030701 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:37.033847 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:37.034584 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:37.034588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:37.034590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:37.034740 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:37.035136 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:37.035140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:37.035143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:37.035469 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:37.035681 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:37.035980 (+ 299us) server_negotiation.cc:299] Negotiation successful
0928 16:36:37.036189 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 16:36:37.037335 12003 generic_service.cc:196] Checking for leaks (request via RPC)
W20260928 16:36:37.093603 12145 tablet.cc:2369] T 698e26666be14ab2b0d5d61352ab20c1 P 57f7bf387e49444694320787bda3160d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:36:37.134935 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 11878
I20260928 16:36:37.152254 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12014
Sep 28 16:36:37 dist-test-slave-9fn6 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:36:37.170763 12294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:37.153375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33062 (local address 127.10.247.194:33125)
0928 16:36:37.153579 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:37.153584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:37.153606 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:36:37.153782 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:37.153786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:37.153872 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:37.153974 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:37.155772 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:37.156912 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:37.159123 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:37.159685 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:37.159806 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:37.164373 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:37.164396 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:37.164418 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:37.164469 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:37.167855 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:37.168580 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:37.168590 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:37.168592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:37.168782 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:37.169231 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:37.169235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:37.169238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:37.169707 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:37.169984 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:37.170231 (+ 247us) server_negotiation.cc:299] Negotiation successful
0928 16:36:37.170438 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:36:37.171523 12138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:37.267550 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 12014
I20260928 16:36:37.284261 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12150
Sep 28 16:36:37 dist-test-slave-9fn6 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:36:37.301199 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:37.285216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40294 (local address 127.10.247.195:36617)
0928 16:36:37.285784 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:37.285790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:37.285807 (+ 17us) server_negotiation.cc:407] Connection header received
0928 16:36:37.285875 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:37.285879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:37.285951 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:37.286074 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:37.287526 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:37.288397 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:37.289831 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:37.290362 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:37.290475 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:37.294976 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:37.295023 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:37.295045 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:37.295088 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:37.298216 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:37.298923 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:37.298931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:37.298933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:37.299114 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:37.299514 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:37.299517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:37.299520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:37.299917 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:37.300200 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:37.300447 (+ 247us) server_negotiation.cc:299] Negotiation successful
0928 16:36:37.300783 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":281,"threads_started":1}
I20260928 16:36:37.302119 12273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:37.399534 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 12150
I20260928 16:36:37.416265 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 11783
I20260928 16:36:37.417757 11845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:37.538759 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 11783
2026-09-28T16:36:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4478 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:36:37 dist-test-slave-9fn6 krb5kdc[12328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:36:37 dist-test-slave-9fn6 krb5kdc[12328](info): set up 2 sockets
Sep 28 16:36:37 dist-test-slave-9fn6 krb5kdc[12328](info): commencing operation
krb5kdc: starting...
W20260928 16:36:38.556844 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.006s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:36:38 dist-test-slave-9fn6 krb5kdc[12328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:36:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:36:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:36:38.776906 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42209
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:46635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:42209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:39.151999 12341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:39.152381 12341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:39.152515 12341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:39.160437 12341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:36:39.160565 12341 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 16:36:39.160638 12341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:39.160734 12341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:39.160799 12341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:36:39.160892 12341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:36:39.171957 12341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:42209
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:39.173731 12341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:39.175460 12341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:39.189010 12346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:39.189036 12349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:39.189059 12347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:39.189858 12341 server_base.cc:1034] running on GCE node
I20260928 16:36:39.190755 12341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:39.192507 12341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:39.193749 12341 hybrid_clock.cc:648] HybridClock initialized: now 1790613399193677 us; error 39 us; skew 500 ppm
I20260928 16:36:39.194150 12341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:39 dist-test-slave-9fn6 krb5kdc[12328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613399, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:39.197770 12341 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:36:39.199563 12341 webserver.cc:460] Webserver started at http://127.10.247.254:45993/ using document root <none> and password file <none>
I20260928 16:36:39.200433 12341 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:39.200531 12341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:39.200918 12341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:39.204556 12341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "4d3c8083ebdb49a7ac42474cca0dc68c"
format_stamp: "Formatted at 2026-09-28 16:36:39 on dist-test-slave-9fn6"
server_key: "1f3e980baf223e430fc1c33c64322d5a"
server_key_iv: "c3040c22dee4692c97c359ab8552d734"
server_key_version: "encryptionkey@0"
I20260928 16:36:39.205529 12341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "4d3c8083ebdb49a7ac42474cca0dc68c"
format_stamp: "Formatted at 2026-09-28 16:36:39 on dist-test-slave-9fn6"
server_key: "1f3e980baf223e430fc1c33c64322d5a"
server_key_iv: "c3040c22dee4692c97c359ab8552d734"
server_key_version: "encryptionkey@0"
I20260928 16:36:39.212214 12341 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260928 16:36:39.216598 12356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:39.218277 12341 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260928 16:36:39.218464 12341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "4d3c8083ebdb49a7ac42474cca0dc68c"
format_stamp: "Formatted at 2026-09-28 16:36:39 on dist-test-slave-9fn6"
server_key: "1f3e980baf223e430fc1c33c64322d5a"
server_key_iv: "c3040c22dee4692c97c359ab8552d734"
server_key_version: "encryptionkey@0"
I20260928 16:36:39.218739 12341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:39.242844 12341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:39.248080 12341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:39.248390 12341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:39.271835 12341 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:42209
I20260928 16:36:39.271857 12417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:42209 every 8 connection(s)
I20260928 16:36:39.273591 12341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:36:39.278299 12418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:39.280500 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 12341
I20260928 16:36:39.280791 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:36:39.281692 11231 external_mini_cluster.cc:1428] Setting key 3514b2218508146925ebe9164e180770
I20260928 16:36:39.292265 12418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c: Bootstrap starting.
I20260928 16:36:39.296710 12418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c: Neither blocks nor log segments found. Creating new log.
Sep 28 16:36:39 dist-test-slave-9fn6 krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:39.298035 12418 log.cc:824] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:39.302351 12418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c: No bootstrap required, opened a new log
I20260928 16:36:39.308775 12421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:39.284113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56334 (local address 127.10.247.254:42209)
0928 16:36:39.285031 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:39.285045 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:39.285098 (+ 53us) server_negotiation.cc:407] Connection header received
0928 16:36:39.286978 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:39.287002 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:39.287453 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:39.287793 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:39.289659 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.291038 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.292794 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.293502 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.293634 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:39.298847 (+ 5213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:39.298859 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:39.298881 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:39.298947 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:39.303058 (+ 4111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:39.304034 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:39.304046 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:39.304057 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:39.304328 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:39.304832 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:39.304838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:39.304841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:39.305744 (+ 903us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:39.306127 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:39.306477 (+ 350us) server_negotiation.cc:299] Negotiation successful
0928 16:36:39.307070 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":428,"threads_started":1}
I20260928 16:36:39.312793 12418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3c8083ebdb49a7ac42474cca0dc68c" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42209 } }
I20260928 16:36:39.313261 12418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:39.313351 12418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d3c8083ebdb49a7ac42474cca0dc68c, State: Initialized, Role: FOLLOWER
I20260928 16:36:39.314045 12418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3c8083ebdb49a7ac42474cca0dc68c" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42209 } }
I20260928 16:36:39.314355 12418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:36:39.314498 12418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:36:39.314672 12418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:39.318015 12418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3c8083ebdb49a7ac42474cca0dc68c" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42209 } }
I20260928 16:36:39.318835 12418 leader_election.cc:302] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d3c8083ebdb49a7ac42474cca0dc68c; no voters:
I20260928 16:36:39.319578 12418 leader_election.cc:288] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:36:39.319744 12423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:39.321183 12423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [term 1 LEADER]: Becoming Leader. State: Replica: 4d3c8083ebdb49a7ac42474cca0dc68c, State: Running, Role: LEADER
I20260928 16:36:39.321884 12423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3c8083ebdb49a7ac42474cca0dc68c" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42209 } }
I20260928 16:36:39.322304 12418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:36:39.326622 12425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d3c8083ebdb49a7ac42474cca0dc68c. Latest consensus state: current_term: 1 leader_uuid: "4d3c8083ebdb49a7ac42474cca0dc68c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3c8083ebdb49a7ac42474cca0dc68c" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42209 } } }
I20260928 16:36:39.327077 12425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:39.326931 12424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d3c8083ebdb49a7ac42474cca0dc68c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3c8083ebdb49a7ac42474cca0dc68c" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42209 } } }
I20260928 16:36:39.327237 12424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:39.329476 12433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:36:39.334885 12433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:36:39.344276 12433 catalog_manager.cc:1347] Generated new cluster ID: f76c2b2051704aaaa113803716502b56
I20260928 16:36:39.344415 12433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:36:39.366570 12433 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:36:39.368949 12433 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:36:39.382210 12433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c: Generated new TSK 0
I20260928 16:36:39.383656 12433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:36:39.436583 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42209
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:46635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:39.809970 12444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:39.810303 12444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:39.810420 12444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:39.818289 12444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:39.818425 12444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:39.818524 12444 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 16:36:39.818611 12444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:36:39.829772 12444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.247.254:42209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:39.831435 12444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:39.833071 12444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:39.848325 12452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:39.848352 12450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:39.848678 12449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:39.849459 12444 server_base.cc:1034] running on GCE node
I20260928 16:36:39.850154 12444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:39.851226 12444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:39.852451 12444 hybrid_clock.cc:648] HybridClock initialized: now 1790613399852371 us; error 57 us; skew 500 ppm
I20260928 16:36:39.852825 12444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:39 dist-test-slave-9fn6 krb5kdc[12328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613399, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:39.859830 12444 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:36:39.861627 12444 webserver.cc:460] Webserver started at http://127.10.247.193:46815/ using document root <none> and password file <none>
I20260928 16:36:39.862474 12444 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:39.862598 12444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:39.862952 12444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:39.866546 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "7986054924fa40e5b7e3263a45183623"
format_stamp: "Formatted at 2026-09-28 16:36:39 on dist-test-slave-9fn6"
server_key: "4aab855653a625455b804bdda0b68ff6"
server_key_iv: "95bc6c54cc2eed2489c411ae47f5e888"
server_key_version: "encryptionkey@0"
I20260928 16:36:39.867441 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "7986054924fa40e5b7e3263a45183623"
format_stamp: "Formatted at 2026-09-28 16:36:39 on dist-test-slave-9fn6"
server_key: "4aab855653a625455b804bdda0b68ff6"
server_key_iv: "95bc6c54cc2eed2489c411ae47f5e888"
server_key_version: "encryptionkey@0"
I20260928 16:36:39.873687 12444 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260928 16:36:39.877772 12459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:39.879315 12444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260928 16:36:39.879492 12444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "7986054924fa40e5b7e3263a45183623"
format_stamp: "Formatted at 2026-09-28 16:36:39 on dist-test-slave-9fn6"
server_key: "4aab855653a625455b804bdda0b68ff6"
server_key_iv: "95bc6c54cc2eed2489c411ae47f5e888"
server_key_version: "encryptionkey@0"
I20260928 16:36:39.879746 12444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:39.901368 12444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:39.906068 12444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:39.906371 12444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:39.909610 12444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:39.909725 12444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:39.909869 12444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:39.909957 12444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:36:39 dist-test-slave-9fn6 krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613399, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:39.937536 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:39.910278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:36175 (local address 127.10.247.254:42209)
0928 16:36:39.911014 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:39.911019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:39.912133 (+ 1114us) server_negotiation.cc:407] Connection header received
0928 16:36:39.914419 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:39.914426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:39.914539 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:39.914687 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:39.917626 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.918674 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.921749 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.922328 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.922475 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:39.928093 (+ 5618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:39.928103 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:39.928108 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:39.928168 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:39.931938 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:39.933156 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:39.933163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:39.933166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:39.933339 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:39.933987 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:39.934004 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:39.934008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:39.934332 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:39.934479 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:39.936868 (+ 2389us) server_negotiation.cc:299] Negotiation successful
0928 16:36:39.937158 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":466,"threads_started":1}
I20260928 16:36:39.938413 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:39.911038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42209 (local address 127.10.247.193:36175)
0928 16:36:39.911944 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:39.911988 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:39.914093 (+ 2105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:39.915043 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:39.915055 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:39.916104 (+ 1049us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:39.917313 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:39.917348 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.918889 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.918895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:39.921553 (+ 2658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:39.921573 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.922643 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.922651 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:39.922963 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:39.923940 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:39.923978 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:39.927686 (+ 3708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:39.932182 (+ 4496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:39.932198 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:39.932224 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:39.932960 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:39.933493 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:39.933498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:39.933502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:39.933823 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:39.934564 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:39.934574 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:39.935215 (+ 641us) client_negotiation.cc:769] Sending connection context
0928 16:36:39.935592 (+ 377us) client_negotiation.cc:240] Negotiation successful
0928 16:36:39.936373 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":387,"threads_started":1}
I20260928 16:36:39.959874 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:39.945593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:35487 (local address 127.10.247.254:42209)
0928 16:36:39.945805 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:39.945811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:39.946635 (+ 824us) server_negotiation.cc:407] Connection header received
0928 16:36:39.946887 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:39.946894 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:39.946977 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:39.947083 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:39.948878 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.949677 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.951076 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.951559 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.951682 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:39.953879 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:39.953885 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:39.953888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:39.953926 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:39.956783 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:39.957843 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:39.957848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:39.957850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:39.957994 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:39.958443 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:39.958449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:39.958452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:39.958782 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:39.958914 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:39.959393 (+ 479us) server_negotiation.cc:299] Negotiation successful
0928 16:36:39.959595 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 16:36:39.960240 12523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:39.945709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42209 (local address 127.10.247.193:35487)
0928 16:36:39.946488 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:39.946523 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:39.946720 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:39.947308 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:39.947314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:39.948134 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:39.948715 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:39.948730 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.949824 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.949830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:39.950926 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:39.950939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:39.951761 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:39.951768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:39.951952 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:39.952812 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:39.952832 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:39.953672 (+ 840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:39.956929 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:39.956936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:39.956944 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:39.957710 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:39.958113 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:39.958118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:39.958122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:39.958318 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:39.958887 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:39.958892 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:39.959136 (+ 244us) client_negotiation.cc:769] Sending connection context
0928 16:36:39.959296 (+ 160us) client_negotiation.cc:240] Negotiation successful
0928 16:36:39.959727 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":426,"threads_started":1}
I20260928 16:36:39.974852 12444 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:35659
I20260928 16:36:39.974867 12579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:35659 every 8 connection(s)
I20260928 16:36:39.976430 12444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:36:39.978991 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 12444
I20260928 16:36:39.979257 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:36:39.979314 12580 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42209
I20260928 16:36:39.979619 12580 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:39.980242 11231 external_mini_cluster.cc:1428] Setting key 6081af7c798c0f6f71aa61f78a9ca5dc
I20260928 16:36:39.980727 12580 heartbeater.cc:511] Master 127.10.247.254:42209 requested a full tablet report, sending...
I20260928 16:36:39.983495 12372 ts_manager.cc:194] Registered new tserver with Master: 7986054924fa40e5b7e3263a45183623 (127.10.247.193:35659)
I20260928 16:36:39.985715 12372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:36175
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:36:40.030736 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42209
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:46635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:40.418407 12584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:40.418790 12584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:40.418929 12584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:40.427196 12584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:40.427371 12584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:40.427501 12584 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 16:36:40.427623 12584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:36:40.439045 12584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.247.254:42209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:40.440938 12584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:40.442776 12584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:40.457805 12589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:40.457877 12590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:40.457849 12592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:40.458899 12584 server_base.cc:1034] running on GCE node
I20260928 16:36:40.459661 12584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:40.460937 12584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:40.462173 12584 hybrid_clock.cc:648] HybridClock initialized: now 1790613400462092 us; error 55 us; skew 500 ppm
I20260928 16:36:40.462512 12584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:40 dist-test-slave-9fn6 krb5kdc[12328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613400, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:40.466231 12584 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:36:40.467926 12584 webserver.cc:460] Webserver started at http://127.10.247.194:33625/ using document root <none> and password file <none>
I20260928 16:36:40.468786 12584 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:40.468885 12584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:40.469220 12584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:40.472695 12584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "33a277a99edb44af9aa3181e63a87068"
format_stamp: "Formatted at 2026-09-28 16:36:40 on dist-test-slave-9fn6"
server_key: "329a35f40a3d1b7bf3202fec303c4291"
server_key_iv: "ee50713a916f94aacbdf1af0a2391575"
server_key_version: "encryptionkey@0"
I20260928 16:36:40.473572 12584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "33a277a99edb44af9aa3181e63a87068"
format_stamp: "Formatted at 2026-09-28 16:36:40 on dist-test-slave-9fn6"
server_key: "329a35f40a3d1b7bf3202fec303c4291"
server_key_iv: "ee50713a916f94aacbdf1af0a2391575"
server_key_version: "encryptionkey@0"
I20260928 16:36:40.479633 12584 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:36:40.483765 12599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:40.485463 12584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:36:40.485618 12584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "33a277a99edb44af9aa3181e63a87068"
format_stamp: "Formatted at 2026-09-28 16:36:40 on dist-test-slave-9fn6"
server_key: "329a35f40a3d1b7bf3202fec303c4291"
server_key_iv: "ee50713a916f94aacbdf1af0a2391575"
server_key_version: "encryptionkey@0"
I20260928 16:36:40.485848 12584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:40.518680 12584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:40.524084 12584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:40.524402 12584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:40.527763 12584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:40.527856 12584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:40.527971 12584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:40.528039 12584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:36:40 dist-test-slave-9fn6 krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613400, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:40.554774 12608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:40.528729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:52047 (local address 127.10.247.254:42209)
0928 16:36:40.529253 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:40.529260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:40.531129 (+ 1869us) server_negotiation.cc:407] Connection header received
0928 16:36:40.534560 (+ 3431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:40.534568 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:40.534692 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:40.534835 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:40.537595 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:40.539007 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:40.541630 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:40.542154 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:40.542289 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:40.547215 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:40.547222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:40.547225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:40.547272 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:40.550587 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:40.551679 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:40.551687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:40.551691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:40.551899 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:40.552579 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:40.552586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:40.552589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:40.552975 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:40.553150 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:40.554044 (+ 894us) server_negotiation.cc:299] Negotiation successful
0928 16:36:40.554331 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":1432,"threads_started":1}
I20260928 16:36:40.556553 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:40.529838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42209 (local address 127.10.247.194:52047)
0928 16:36:40.530885 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:40.530953 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:40.534069 (+ 3116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:40.535205 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:40.535216 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:40.536158 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:40.537279 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:40.537323 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:40.539201 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:40.539205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:40.541371 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:40.541393 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:40.542433 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:40.542441 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:40.542698 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:40.543628 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:40.543658 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:40.546887 (+ 3229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:40.550764 (+ 3877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:40.550778 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:40.550800 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:40.551507 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:40.552043 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:40.552049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:40.552053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:40.552380 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:40.553126 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:40.553135 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:40.553617 (+ 482us) client_negotiation.cc:769] Sending connection context
0928 16:36:40.553907 (+ 290us) client_negotiation.cc:240] Negotiation successful
0928 16:36:40.554600 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":421,"threads_started":1}
I20260928 16:36:40.581530 12608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:40.564963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:33309 (local address 127.10.247.254:42209)
0928 16:36:40.565490 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:40.565498 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:40.565992 (+ 494us) server_negotiation.cc:407] Connection header received
0928 16:36:40.566263 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:40.566270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:40.566383 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:40.566514 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:40.568144 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:40.569439 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:40.571004 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:40.571769 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:40.571917 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:40.574496 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:40.574504 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:40.574509 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:40.574561 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:40.577996 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:40.578930 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:40.578936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:40.578940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:40.579130 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:40.579653 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:40.579660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:40.579663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:40.580189 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:40.580343 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:40.580843 (+ 500us) server_negotiation.cc:299] Negotiation successful
0928 16:36:40.581134 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345}
I20260928 16:36:40.581719 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:40.565035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42209 (local address 127.10.247.194:33309)
0928 16:36:40.565836 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:40.565866 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:40.566078 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:40.566579 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:40.566584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:40.567342 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:40.567937 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:40.567957 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:40.569628 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:40.569634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:40.570847 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:40.570871 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:40.572030 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:40.572039 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:40.572264 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:40.573033 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:40.573053 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:40.574151 (+ 1098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:40.578143 (+ 3992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:40.578150 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:40.578155 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:40.578768 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:40.579322 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:40.579327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:40.579331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:40.579519 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:40.580314 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:40.580319 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:40.580539 (+ 220us) client_negotiation.cc:769] Sending connection context
0928 16:36:40.580739 (+ 200us) client_negotiation.cc:240] Negotiation successful
0928 16:36:40.581205 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":455,"threads_started":1}
I20260928 16:36:40.582643 12584 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:37431
I20260928 16:36:40.582664 12720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:37431 every 8 connection(s)
I20260928 16:36:40.584470 12584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:36:40.586726 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 12584
I20260928 16:36:40.586987 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:36:40.587673 11231 external_mini_cluster.cc:1428] Setting key 18b01fde20173151d90a05c61a1668bb
I20260928 16:36:40.589283 12721 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42209
I20260928 16:36:40.589614 12721 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:40.590667 12721 heartbeater.cc:511] Master 127.10.247.254:42209 requested a full tablet report, sending...
I20260928 16:36:40.592327 12373 ts_manager.cc:194] Registered new tserver with Master: 33a277a99edb44af9aa3181e63a87068 (127.10.247.194:37431)
I20260928 16:36:40.593403 12373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:52047
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:36:40.636390 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42209
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:46635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:36:40.990530 12580 heartbeater.cc:503] Master 127.10.247.254:42209 was elected leader, sending a full tablet report...
W20260928 16:36:41.021339 12725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:41.021781 12725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:41.021937 12725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:41.030284 12725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:41.030431 12725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:41.030555 12725 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 16:36:41.030661 12725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:36:41.043562 12725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.247.254:42209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:41.046065 12725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:41.048102 12725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:41.063414 12730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:41.063455 12733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:41.063481 12731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:41.064363 12725 server_base.cc:1034] running on GCE node
I20260928 16:36:41.065155 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:41.066265 12725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:41.067509 12725 hybrid_clock.cc:648] HybridClock initialized: now 1790613401067431 us; error 35 us; skew 500 ppm
I20260928 16:36:41.067852 12725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:41 dist-test-slave-9fn6 krb5kdc[12328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613401, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:41.071426 12725 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:36:41.073236 12725 webserver.cc:460] Webserver started at http://127.10.247.195:45887/ using document root <none> and password file <none>
I20260928 16:36:41.074062 12725 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:41.074156 12725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:41.074470 12725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:41.077970 12725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "f1bbd18e041f4252b03a4e09b9256a2a"
format_stamp: "Formatted at 2026-09-28 16:36:41 on dist-test-slave-9fn6"
server_key: "e305dbcf209e4c656407dc0b7a1acb42"
server_key_iv: "ce792666ae4909cad90e41ded90f9f52"
server_key_version: "encryptionkey@0"
I20260928 16:36:41.078881 12725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "f1bbd18e041f4252b03a4e09b9256a2a"
format_stamp: "Formatted at 2026-09-28 16:36:41 on dist-test-slave-9fn6"
server_key: "e305dbcf209e4c656407dc0b7a1acb42"
server_key_iv: "ce792666ae4909cad90e41ded90f9f52"
server_key_version: "encryptionkey@0"
I20260928 16:36:41.085194 12725 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260928 16:36:41.089689 12740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:41.091511 12725 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:36:41.091703 12725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "f1bbd18e041f4252b03a4e09b9256a2a"
format_stamp: "Formatted at 2026-09-28 16:36:41 on dist-test-slave-9fn6"
server_key: "e305dbcf209e4c656407dc0b7a1acb42"
server_key_iv: "ce792666ae4909cad90e41ded90f9f52"
server_key_version: "encryptionkey@0"
I20260928 16:36:41.091962 12725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:41.116592 12725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:41.121696 12725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:41.122000 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:41.125406 12725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:41.125510 12725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:41.125639 12725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:41.125696 12725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:36:41 dist-test-slave-9fn6 krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613401, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:41.153713 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.126221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:52091 (local address 127.10.247.254:42209)
0928 16:36:41.126775 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.126783 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.128257 (+ 1474us) server_negotiation.cc:407] Connection header received
0928 16:36:41.130928 (+ 2671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.130935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.131052 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.131174 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:41.133753 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.135052 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.137854 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.138563 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.138718 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.144557 (+ 5839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:41.144566 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:41.144573 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:41.144660 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:41.148596 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:41.149919 (+ 1323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:41.149928 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:41.149932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:41.150158 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:41.150892 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:41.150898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:41.150902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:41.151372 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:41.151552 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.152939 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.153265 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":252,"threads_started":1}
I20260928 16:36:41.155534 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.127058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42209 (local address 127.10.247.195:52091)
0928 16:36:41.128037 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:41.128095 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:41.130430 (+ 2335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:41.131498 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:41.131508 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:41.132446 (+ 938us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:41.133473 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.133509 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.135245 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.135251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.137560 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.137586 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.138890 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.138899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.139280 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.140323 (+ 1043us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:41.140362 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:41.144138 (+ 3776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:41.148826 (+ 4688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:41.148835 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:41.148854 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:41.149727 (+ 873us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:41.150350 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:41.150356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:41.150360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:41.150726 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:41.151533 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:41.151548 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:41.152334 (+ 786us) client_negotiation.cc:769] Sending connection context
0928 16:36:41.152791 (+ 457us) client_negotiation.cc:240] Negotiation successful
0928 16:36:41.153493 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":392,"threads_started":1}
I20260928 16:36:41.168736 12725 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:39967
I20260928 16:36:41.168793 12863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:39967 every 8 connection(s)
I20260928 16:36:41.170531 12725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:36:41.173146 12864 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42209
I20260928 16:36:41.173442 12864 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:41.174324 12864 heartbeater.cc:511] Master 127.10.247.254:42209 requested a full tablet report, sending...
I20260928 16:36:41.176229 12373 ts_manager.cc:194] Registered new tserver with Master: f1bbd18e041f4252b03a4e09b9256a2a (127.10.247.195:39967)
I20260928 16:36:41.177384 12373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:52091
I20260928 16:36:41.180929 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 12725
I20260928 16:36:41.181169 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:36:41.181797 11231 external_mini_cluster.cc:1428] Setting key c92ff1e50ab4664f4e2df6215030e168
I20260928 16:36:41.181990 12862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.167545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42209 (local address 127.10.247.195:39885)
0928 16:36:41.168370 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:41.168392 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:41.168634 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:41.169059 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:41.169063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:41.169560 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:41.169891 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.169902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.171435 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.171441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.173160 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.173178 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.174100 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.174107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.174298 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.174981 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:41.175005 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:41.175947 (+ 942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:41.179398 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:41.179403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:41.179408 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:41.179866 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:41.180252 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:41.180258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:41.180262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:41.180406 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:41.180994 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:41.180999 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:41.181187 (+ 188us) client_negotiation.cc:769] Sending connection context
0928 16:36:41.181327 (+ 140us) client_negotiation.cc:240] Negotiation successful
0928 16:36:41.181613 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":488,"threads_started":1}
I20260928 16:36:41.182096 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.167542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:39885 (local address 127.10.247.254:42209)
0928 16:36:41.167786 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.167793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.168534 (+ 741us) server_negotiation.cc:407] Connection header received
0928 16:36:41.168800 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.168806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.168912 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.169029 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:41.170063 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.171265 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.173355 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.173888 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.173999 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.176173 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:41.176178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:41.176181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:41.176222 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:41.179252 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:41.179989 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:41.179994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:41.179996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:41.180139 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:41.180507 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:41.180513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:41.180515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:41.180865 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:41.181024 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.181455 (+ 431us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.181722 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260928 16:36:41.185521 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 16:36:41 dist-test-slave-9fn6 krb5kdc[12328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613401, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 16:36:41 dist-test-slave-9fn6 krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613401, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:41.238206 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.222567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.10.247.254:42209)
0928 16:36:41.222878 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.222885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.223088 (+ 203us) server_negotiation.cc:407] Connection header received
0928 16:36:41.223307 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.223313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.223417 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.223513 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:41.225171 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.226082 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.227449 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.227943 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.228068 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.232177 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:41.232182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:41.232186 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:41.232237 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:41.235115 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:41.235918 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:41.235924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:41.235926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:41.236132 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:41.236556 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:41.236560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:41.236562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:41.237049 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:41.237282 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.237574 (+ 292us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.237793 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260928 16:36:41.243149 12373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:36:41.251451 12373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:36:41.306150 12882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.269168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42474 (local address 127.10.247.194:37431)
0928 16:36:41.294920 (+ 25752us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.294930 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.297014 (+ 2084us) server_negotiation.cc:407] Connection header received
0928 16:36:41.297455 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.297463 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.297691 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.297892 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:41.298893 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.300336 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.303163 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.304453 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.304593 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.304861 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.305593 (+ 732us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.305813 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25539,"thread_start_us":25753,"threads_started":1}
I20260928 16:36:41.306242 12880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.267227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:39967 (local address 127.0.0.1:58126)
0928 16:36:41.291390 (+ 24163us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:41.291475 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:41.291860 (+ 385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:41.297201 (+ 5341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:41.297207 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:41.297236 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:41.297696 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.297708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.300049 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.300055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.302800 (+ 2745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.302816 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.304932 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.304939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.305192 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.305303 (+ 111us) client_negotiation.cc:769] Sending connection context
0928 16:36:41.305467 (+ 164us) client_negotiation.cc:240] Negotiation successful
0928 16:36:41.305775 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23695,"mutex_wait_us":81,"thread_start_us":29241,"threads_started":1}
I20260928 16:36:41.306326 12878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.267450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.10.247.195:39967)
0928 16:36:41.296398 (+ 28948us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.296409 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.296454 (+ 45us) server_negotiation.cc:407] Connection header received
0928 16:36:41.296567 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.296575 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.296940 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.297888 (+ 948us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:41.298234 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.299889 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.303026 (+ 3137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.304698 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.304914 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.305108 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.305732 (+ 624us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.305932 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28725,"thread_start_us":28439,"threads_started":1}
I20260928 16:36:41.306418 12879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.267662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:35659 (local address 127.0.0.1:34846)
0928 16:36:41.293653 (+ 25991us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:41.293673 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:41.293818 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:41.294996 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:41.295005 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:41.295039 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:41.295558 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.295573 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.299691 (+ 4118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.299696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.303353 (+ 3657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.303367 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.305485 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.305492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.305732 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.305766 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 16:36:41.305854 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 16:36:41.306006 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25792,"mutex_wait_us":119,"thread_start_us":25642,"threads_started":1}
I20260928 16:36:41.307175 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.268548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.10.247.193:35659)
0928 16:36:41.291574 (+ 23026us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.291588 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.293963 (+ 2375us) server_negotiation.cc:407] Connection header received
0928 16:36:41.294096 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.294105 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.294444 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.294678 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:41.295825 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.299456 (+ 3631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.303584 (+ 4128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.305228 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.305415 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.305602 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.306670 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.306873 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22094,"thread_start_us":21561,"threads_started":1}
I20260928 16:36:41.307875 12877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.268014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:37431 (local address 127.0.0.1:42474)
0928 16:36:41.296859 (+ 28845us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:41.296880 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:41.297140 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:41.298183 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:41.298187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:41.298213 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:41.298685 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.298697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.300551 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.300557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.302895 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.302908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.305064 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.305070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.305250 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.305302 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 16:36:41.307396 (+ 2094us) client_negotiation.cc:240] Negotiation successful
0928 16:36:41.307535 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28689,"mutex_wait_us":93,"thread_start_us":24249,"threads_started":1}
I20260928 16:36:41.311378 12650 tablet_service.cc:1467] Processing CreateTablet for tablet 2d2d29ab4bff45268c24dd6a35e0cf26 (DEFAULT_TABLE table=test-table [id=9bd148f0d3874f0d8ad46a8059ef9ba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:41.311409 12510 tablet_service.cc:1467] Processing CreateTablet for tablet 2d2d29ab4bff45268c24dd6a35e0cf26 (DEFAULT_TABLE table=test-table [id=9bd148f0d3874f0d8ad46a8059ef9ba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:41.311393 12793 tablet_service.cc:1467] Processing CreateTablet for tablet 2d2d29ab4bff45268c24dd6a35e0cf26 (DEFAULT_TABLE table=test-table [id=9bd148f0d3874f0d8ad46a8059ef9ba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:41.312930 12650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d2d29ab4bff45268c24dd6a35e0cf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:41.312947 12793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d2d29ab4bff45268c24dd6a35e0cf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:41.312947 12510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d2d29ab4bff45268c24dd6a35e0cf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:41.326985 12883 tablet_bootstrap.cc:492] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: Bootstrap starting.
I20260928 16:36:41.329427 12884 tablet_bootstrap.cc:492] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: Bootstrap starting.
I20260928 16:36:41.331485 12883 tablet_bootstrap.cc:654] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:41.333280 12883 log.cc:824] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:41.333549 12884 tablet_bootstrap.cc:654] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:41.334851 12884 log.cc:824] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:41.335095 12885 tablet_bootstrap.cc:492] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: Bootstrap starting.
I20260928 16:36:41.337576 12883 tablet_bootstrap.cc:492] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: No bootstrap required, opened a new log
I20260928 16:36:41.337913 12883 ts_tablet_manager.cc:1397] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260928 16:36:41.338382 12884 tablet_bootstrap.cc:492] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: No bootstrap required, opened a new log
I20260928 16:36:41.338687 12884 ts_tablet_manager.cc:1397] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260928 16:36:41.340344 12885 tablet_bootstrap.cc:654] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:41.341904 12885 log.cc:824] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:41.346599 12883 raft_consensus.cc:348] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } }
I20260928 16:36:41.347059 12883 raft_consensus.cc:374] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:41.347198 12883 raft_consensus.cc:729] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a277a99edb44af9aa3181e63a87068, State: Initialized, Role: FOLLOWER
I20260928 16:36:41.348244 12883 consensus_queue.cc:260] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } }
I20260928 16:36:41.349592 12721 heartbeater.cc:503] Master 127.10.247.254:42209 was elected leader, sending a full tablet report...
I20260928 16:36:41.350143 12883 ts_tablet_manager.cc:1428] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260928 16:36:41.350735 12885 tablet_bootstrap.cc:492] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: No bootstrap required, opened a new log
I20260928 16:36:41.351102 12885 ts_tablet_manager.cc:1397] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260928 16:36:41.353192 12884 raft_consensus.cc:348] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } }
I20260928 16:36:41.353767 12884 raft_consensus.cc:374] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:41.353886 12884 raft_consensus.cc:729] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7986054924fa40e5b7e3263a45183623, State: Initialized, Role: FOLLOWER
I20260928 16:36:41.354846 12884 consensus_queue.cc:260] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } }
I20260928 16:36:41.364138 12885 raft_consensus.cc:348] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } }
I20260928 16:36:41.365780 12885 raft_consensus.cc:374] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:41.365921 12885 raft_consensus.cc:729] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bbd18e041f4252b03a4e09b9256a2a, State: Initialized, Role: FOLLOWER
I20260928 16:36:41.366729 12884 ts_tablet_manager.cc:1428] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: Time spent starting tablet: real 0.028s user 0.025s sys 0.000s
I20260928 16:36:41.366806 12885 consensus_queue.cc:260] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } }
I20260928 16:36:41.368137 12864 heartbeater.cc:503] Master 127.10.247.254:42209 was elected leader, sending a full tablet report...
I20260928 16:36:41.369022 12885 ts_tablet_manager.cc:1428] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
I20260928 16:36:41.386152 12892 raft_consensus.cc:482] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:36:41.386536 12892 raft_consensus.cc:504] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } }
I20260928 16:36:41.388774 12892 leader_election.cc:288] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7986054924fa40e5b7e3263a45183623 (127.10.247.193:35659), 33a277a99edb44af9aa3181e63a87068 (127.10.247.194:37431)
I20260928 16:36:41.396279 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.389187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:37679 (local address 127.10.247.193:35659)
0928 16:36:41.389436 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.389443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.389756 (+ 313us) server_negotiation.cc:407] Connection header received
0928 16:36:41.389939 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.389945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.390061 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.390181 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:41.390797 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.391803 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.394230 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.395306 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.395508 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.395596 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.395850 (+ 254us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.396024 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 16:36:41.396298 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.389335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:35659 (local address 127.10.247.195:37679)
0928 16:36:41.389642 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:41.389659 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:41.389797 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:41.390254 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:41.390259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:41.390300 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:41.390645 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.390662 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.391948 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.391953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.394053 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.394070 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.395508 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.395515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.395676 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.395706 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 16:36:41.395807 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 16:36:41.395933 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260928 16:36:41.397064 12882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.389292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:47141 (local address 127.10.247.194:37431)
0928 16:36:41.389519 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.389525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.390694 (+ 1169us) server_negotiation.cc:407] Connection header received
0928 16:36:41.390854 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.390860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.390949 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.391045 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:41.391594 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.392560 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.394688 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.395926 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.396081 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.396181 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.396546 (+ 365us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.396735 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:36:41.397131 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.389651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:37431 (local address 127.10.247.195:47141)
0928 16:36:41.390578 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:41.390597 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:41.390723 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:41.391094 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:41.391097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:41.391117 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:41.391437 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.391446 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.392712 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.392718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.394544 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.394554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.396192 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.396198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.396373 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.396417 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 16:36:41.396507 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 16:36:41.396733 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":503,"threads_started":1}
I20260928 16:36:41.397675 12535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d2d29ab4bff45268c24dd6a35e0cf26" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7986054924fa40e5b7e3263a45183623" is_pre_election: true
I20260928 16:36:41.398339 12535 raft_consensus.cc:2454] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 0.
I20260928 16:36:41.398559 12671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d2d29ab4bff45268c24dd6a35e0cf26" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a277a99edb44af9aa3181e63a87068" is_pre_election: true
I20260928 16:36:41.399186 12671 raft_consensus.cc:2454] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 0.
I20260928 16:36:41.399211 12743 leader_election.cc:302] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7986054924fa40e5b7e3263a45183623, f1bbd18e041f4252b03a4e09b9256a2a; no voters:
I20260928 16:36:41.399834 12892 raft_consensus.cc:2781] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:36:41.400053 12892 raft_consensus.cc:482] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:36:41.400185 12892 raft_consensus.cc:3037] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:41.402897 12892 raft_consensus.cc:504] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } }
I20260928 16:36:41.403637 12892 leader_election.cc:288] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 election: Requested vote from peers 7986054924fa40e5b7e3263a45183623 (127.10.247.193:35659), 33a277a99edb44af9aa3181e63a87068 (127.10.247.194:37431)
I20260928 16:36:41.404291 12535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d2d29ab4bff45268c24dd6a35e0cf26" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7986054924fa40e5b7e3263a45183623"
I20260928 16:36:41.404440 12671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d2d29ab4bff45268c24dd6a35e0cf26" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a277a99edb44af9aa3181e63a87068"
I20260928 16:36:41.404532 12535 raft_consensus.cc:3037] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:41.404763 12671 raft_consensus.cc:3037] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:41.407414 12671 raft_consensus.cc:2454] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 1.
I20260928 16:36:41.407438 12535 raft_consensus.cc:2454] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 1.
I20260928 16:36:41.407959 12741 leader_election.cc:302] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33a277a99edb44af9aa3181e63a87068, f1bbd18e041f4252b03a4e09b9256a2a; no voters:
I20260928 16:36:41.408354 12892 raft_consensus.cc:2781] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:41.408892 12892 raft_consensus.cc:686] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 LEADER]: Becoming Leader. State: Replica: f1bbd18e041f4252b03a4e09b9256a2a, State: Running, Role: LEADER
I20260928 16:36:41.409458 12892 consensus_queue.cc:237] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } }
I20260928 16:36:41.415490 12372 catalog_manager.cc:5495] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bbd18e041f4252b03a4e09b9256a2a (127.10.247.195). New cstate: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } health_report { overall_health: UNKNOWN } } }
W20260928 16:36:41.423024 12865 tablet.cc:2369] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:36:41.470813 12878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.465121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.10.247.195:39967)
0928 16:36:41.465314 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.465319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.465363 (+ 44us) server_negotiation.cc:407] Connection header received
0928 16:36:41.465696 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.465700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.465795 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.465932 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:41.466284 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.467099 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.468465 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.468995 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.469105 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.469599 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:41.470255 (+ 656us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:41.470366 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.470451 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.470549 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260928 16:36:41.479509 12581 tablet.cc:2369] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:36:41.481695 12535 raft_consensus.cc:1261] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [term 1 FOLLOWER]: Refusing update from remote peer f1bbd18e041f4252b03a4e09b9256a2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:41.482764 12894 consensus_queue.cc:1035] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:41.483314 12671 raft_consensus.cc:1261] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [term 1 FOLLOWER]: Refusing update from remote peer f1bbd18e041f4252b03a4e09b9256a2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:41.484213 12894 consensus_queue.cc:1035] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:41.497675 12897 mvcc.cc:205] Tried to move back new op lower bound from 7334352492454998016 to 7334352492177809408. Current Snapshot: MvccSnapshot[applied={T|T < 7334352492454998016}]
I20260928 16:36:41.524358 12373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:41.524854 12373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:41.529675 12373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c: Sending DeleteTablet for 3 replicas of tablet 2d2d29ab4bff45268c24dd6a35e0cf26
I20260928 16:36:41.531271 12793 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d2d29ab4bff45268c24dd6a35e0cf26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:41 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:58126
I20260928 16:36:41.531658 12510 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d2d29ab4bff45268c24dd6a35e0cf26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:41 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:34846
I20260928 16:36:41.532737 12650 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d2d29ab4bff45268c24dd6a35e0cf26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:41 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:42474
I20260928 16:36:41.533879 12373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:36:41.534960 12373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:36:41.538633 12909 tablet_replica.cc:330] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: stopping tablet replica
I20260928 16:36:41.539367 12909 raft_consensus.cc:2229] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:41.540129 12909 raft_consensus.cc:2258] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:41.543236 12909 ts_tablet_manager.cc:1905] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:41.546422 12509 tablet_service.cc:1467] Processing CreateTablet for tablet f5df537754a048dab3a892fa321532a7 (DEFAULT_TABLE table=test-table [id=a22821b72759428aaac61d3a6e177ff3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:41.546525 12792 tablet_service.cc:1467] Processing CreateTablet for tablet f5df537754a048dab3a892fa321532a7 (DEFAULT_TABLE table=test-table [id=a22821b72759428aaac61d3a6e177ff3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:41.547165 12509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5df537754a048dab3a892fa321532a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:41.547183 12792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5df537754a048dab3a892fa321532a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:41.551308 12908 tablet_replica.cc:330] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: stopping tablet replica
I20260928 16:36:41.551935 12908 raft_consensus.cc:2229] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:36:41.552949 12908 raft_consensus.cc:2258] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:41.553098 12650 tablet_service.cc:1467] Processing CreateTablet for tablet f5df537754a048dab3a892fa321532a7 (DEFAULT_TABLE table=test-table [id=a22821b72759428aaac61d3a6e177ff3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:41.553663 12650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5df537754a048dab3a892fa321532a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:41.553930 12907 tablet_replica.cc:330] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: stopping tablet replica
I20260928 16:36:41.554484 12907 raft_consensus.cc:2229] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:41.555256 12884 tablet_bootstrap.cc:492] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: Bootstrap starting.
I20260928 16:36:41.557317 12884 tablet_bootstrap.cc:654] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:41.561338 12907 raft_consensus.cc:2258] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:41.561877 12908 ts_tablet_manager.cc:1905] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:41.565151 12909 ts_tablet_manager.cc:1918] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:36:41.563539 12885 tablet_bootstrap.cc:492] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: Bootstrap starting.
I20260928 16:36:41.566234 12907 ts_tablet_manager.cc:1905] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:41.567425 12885 tablet_bootstrap.cc:654] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:41.570058 12883 tablet_bootstrap.cc:492] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: Bootstrap starting.
I20260928 16:36:41.573418 12884 tablet_bootstrap.cc:492] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: No bootstrap required, opened a new log
I20260928 16:36:41.572345 12885 tablet_bootstrap.cc:492] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: No bootstrap required, opened a new log
I20260928 16:36:41.573752 12884 ts_tablet_manager.cc:1397] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.004s
I20260928 16:36:41.573786 12885 ts_tablet_manager.cc:1397] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260928 16:36:41.574934 12883 tablet_bootstrap.cc:654] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:41.574777 12884 raft_consensus.cc:348] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:41.574839 12885 raft_consensus.cc:348] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:41.575037 12884 raft_consensus.cc:374] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:41.575089 12885 raft_consensus.cc:374] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:41.575132 12884 raft_consensus.cc:729] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7986054924fa40e5b7e3263a45183623, State: Initialized, Role: FOLLOWER
I20260928 16:36:41.575207 12885 raft_consensus.cc:729] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bbd18e041f4252b03a4e09b9256a2a, State: Initialized, Role: FOLLOWER
I20260928 16:36:41.575407 12884 consensus_queue.cc:260] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:41.575505 12885 consensus_queue.cc:260] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:41.576141 12885 ts_tablet_manager.cc:1428] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260928 16:36:41.576823 12884 ts_tablet_manager.cc:1428] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 16:36:41.579491 12909 log.cc:1196] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/2d2d29ab4bff45268c24dd6a35e0cf26
I20260928 16:36:41.582671 12907 ts_tablet_manager.cc:1918] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:36:41.582973 12907 log.cc:1196] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/2d2d29ab4bff45268c24dd6a35e0cf26
I20260928 16:36:41.583706 12907 ts_tablet_manager.cc:1939] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 7986054924fa40e5b7e3263a45183623: Deleting consensus metadata
I20260928 16:36:41.584888 12909 ts_tablet_manager.cc:1939] T 2d2d29ab4bff45268c24dd6a35e0cf26 P 33a277a99edb44af9aa3181e63a87068: Deleting consensus metadata
I20260928 16:36:41.586051 12359 catalog_manager.cc:4841] TS 7986054924fa40e5b7e3263a45183623 (127.10.247.193:35659): tablet 2d2d29ab4bff45268c24dd6a35e0cf26 (table test-table [id=9bd148f0d3874f0d8ad46a8059ef9ba5]) successfully deleted
I20260928 16:36:41.588342 12357 catalog_manager.cc:4841] TS 33a277a99edb44af9aa3181e63a87068 (127.10.247.194:37431): tablet 2d2d29ab4bff45268c24dd6a35e0cf26 (table test-table [id=9bd148f0d3874f0d8ad46a8059ef9ba5]) successfully deleted
I20260928 16:36:41.589780 12883 tablet_bootstrap.cc:492] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: No bootstrap required, opened a new log
I20260928 16:36:41.590054 12883 ts_tablet_manager.cc:1397] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.007s
I20260928 16:36:41.591099 12883 raft_consensus.cc:348] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:41.591457 12883 raft_consensus.cc:374] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:41.591624 12883 raft_consensus.cc:729] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a277a99edb44af9aa3181e63a87068, State: Initialized, Role: FOLLOWER
I20260928 16:36:41.591863 12908 ts_tablet_manager.cc:1918] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:36:41.592087 12908 log.cc:1196] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/2d2d29ab4bff45268c24dd6a35e0cf26
I20260928 16:36:41.592257 12883 consensus_queue.cc:260] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:41.592701 12908 ts_tablet_manager.cc:1939] T 2d2d29ab4bff45268c24dd6a35e0cf26 P f1bbd18e041f4252b03a4e09b9256a2a: Deleting consensus metadata
I20260928 16:36:41.594738 12358 catalog_manager.cc:4841] TS f1bbd18e041f4252b03a4e09b9256a2a (127.10.247.195:39967): tablet 2d2d29ab4bff45268c24dd6a35e0cf26 (table test-table [id=9bd148f0d3874f0d8ad46a8059ef9ba5]) successfully deleted
I20260928 16:36:41.595475 12883 ts_tablet_manager.cc:1428] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
W20260928 16:36:41.843972 12722 tablet.cc:2369] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:36:41.847153 12892 raft_consensus.cc:482] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:36:41.847463 12892 raft_consensus.cc:504] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:41.848481 12892 leader_election.cc:288] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7986054924fa40e5b7e3263a45183623 (127.10.247.193:35659), 33a277a99edb44af9aa3181e63a87068 (127.10.247.194:37431)
I20260928 16:36:41.849292 12671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5df537754a048dab3a892fa321532a7" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a277a99edb44af9aa3181e63a87068" is_pre_election: true
I20260928 16:36:41.849596 12671 raft_consensus.cc:2454] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 0.
I20260928 16:36:41.849380 12535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5df537754a048dab3a892fa321532a7" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7986054924fa40e5b7e3263a45183623" is_pre_election: true
I20260928 16:36:41.849854 12535 raft_consensus.cc:2454] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 0.
I20260928 16:36:41.850139 12741 leader_election.cc:302] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33a277a99edb44af9aa3181e63a87068, f1bbd18e041f4252b03a4e09b9256a2a; no voters:
I20260928 16:36:41.850639 12892 raft_consensus.cc:2781] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:36:41.850740 12892 raft_consensus.cc:482] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:36:41.850862 12892 raft_consensus.cc:3037] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:41.853400 12892 raft_consensus.cc:504] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:41.854368 12892 leader_election.cc:288] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 election: Requested vote from peers 7986054924fa40e5b7e3263a45183623 (127.10.247.193:35659), 33a277a99edb44af9aa3181e63a87068 (127.10.247.194:37431)
I20260928 16:36:41.854939 12535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5df537754a048dab3a892fa321532a7" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7986054924fa40e5b7e3263a45183623"
I20260928 16:36:41.855033 12671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5df537754a048dab3a892fa321532a7" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a277a99edb44af9aa3181e63a87068"
I20260928 16:36:41.855237 12535 raft_consensus.cc:3037] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:41.855268 12671 raft_consensus.cc:3037] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:41.857719 12671 raft_consensus.cc:2454] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 1.
I20260928 16:36:41.858232 12741 leader_election.cc:302] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33a277a99edb44af9aa3181e63a87068, f1bbd18e041f4252b03a4e09b9256a2a; no voters:
I20260928 16:36:41.858675 12535 raft_consensus.cc:2454] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 1.
I20260928 16:36:41.858946 12892 raft_consensus.cc:2781] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:41.859153 12892 raft_consensus.cc:686] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 LEADER]: Becoming Leader. State: Replica: f1bbd18e041f4252b03a4e09b9256a2a, State: Running, Role: LEADER
I20260928 16:36:41.859477 12892 consensus_queue.cc:237] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:41.862973 12371 catalog_manager.cc:5495] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bbd18e041f4252b03a4e09b9256a2a (127.10.247.195). New cstate: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } health_report { overall_health: HEALTHY } } }
Sep 28 16:36:41 dist-test-slave-9fn6 krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613399, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:41.983928 12919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.967679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42209 (local address 127.0.0.1:48764)
0928 16:36:41.968435 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:41.968463 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:41.968699 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:41.969093 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:41.969101 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:41.969949 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:41.970214 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.970226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.971689 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.971693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.972967 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:41.972983 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.973806 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.973812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:41.974024 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.974862 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:41.974883 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:41.977707 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:41.980930 (+ 3223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:41.980948 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:41.980956 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:41.981575 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:41.982018 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:41.982021 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:41.982024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:41.982180 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:41.982777 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:41.982786 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:41.982975 (+ 189us) client_negotiation.cc:769] Sending connection context
0928 16:36:41.983122 (+ 147us) client_negotiation.cc:240] Negotiation successful
0928 16:36:41.983454 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":368,"threads_started":1}
I20260928 16:36:41.984522 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:41.967713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48764 (local address 127.10.247.254:42209)
0928 16:36:41.968798 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:41.968804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:41.968828 (+ 24us) server_negotiation.cc:407] Connection header received
0928 16:36:41.968882 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:41.968885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:41.968945 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:41.969043 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:41.970383 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.971527 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.973112 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:41.973620 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:41.973750 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:41.977927 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:41.977932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:41.977935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:41.977971 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:41.980761 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:41.981712 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:41.981717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:41.981720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:41.981845 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:41.982289 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:41.982293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:41.982295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:41.982626 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:41.982783 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:41.983246 (+ 463us) server_negotiation.cc:299] Negotiation successful
0928 16:36:41.983672 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":189,"server-negotiator.queue_time_us":952,"thread_start_us":770,"threads_started":1}
I20260928 16:36:41.991221 12371 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:48764:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 16:36:41.992537 12371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:36:42.002095 12510 tablet_service.cc:1467] Processing CreateTablet for tablet 9193653597eb4f84b3d6d070e8858d35 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d21c798e1e494f6798734b4620a67442]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 16:36:42.002588 12650 tablet_service.cc:1467] Processing CreateTablet for tablet 9193653597eb4f84b3d6d070e8858d35 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d21c798e1e494f6798734b4620a67442]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 16:36:42.002897 12510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9193653597eb4f84b3d6d070e8858d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:42.004061 12650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9193653597eb4f84b3d6d070e8858d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:42.004251 12792 tablet_service.cc:1467] Processing CreateTablet for tablet 9193653597eb4f84b3d6d070e8858d35 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d21c798e1e494f6798734b4620a67442]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 16:36:42.004851 12792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9193653597eb4f84b3d6d070e8858d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:42.013010 12883 tablet_bootstrap.cc:492] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: Bootstrap starting.
I20260928 16:36:42.014263 12885 tablet_bootstrap.cc:492] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: Bootstrap starting.
I20260928 16:36:42.014724 12884 tablet_bootstrap.cc:492] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: Bootstrap starting.
I20260928 16:36:42.015305 12883 tablet_bootstrap.cc:654] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:42.016539 12885 tablet_bootstrap.cc:654] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:42.016836 12884 tablet_bootstrap.cc:654] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:42.026859 12884 tablet_bootstrap.cc:492] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: No bootstrap required, opened a new log
I20260928 16:36:42.026859 12883 tablet_bootstrap.cc:492] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: No bootstrap required, opened a new log
I20260928 16:36:42.027078 12883 ts_tablet_manager.cc:1397] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260928 16:36:42.027505 12885 tablet_bootstrap.cc:492] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: No bootstrap required, opened a new log
I20260928 16:36:42.027747 12885 ts_tablet_manager.cc:1397] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260928 16:36:42.027220 12884 ts_tablet_manager.cc:1397] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260928 16:36:42.028636 12885 raft_consensus.cc:348] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:42.028998 12885 raft_consensus.cc:374] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:42.029103 12885 raft_consensus.cc:729] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bbd18e041f4252b03a4e09b9256a2a, State: Initialized, Role: FOLLOWER
I20260928 16:36:42.029418 12885 consensus_queue.cc:260] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:42.029407 12884 raft_consensus.cc:348] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:42.029460 12883 raft_consensus.cc:348] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:42.029738 12884 raft_consensus.cc:374] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:42.029844 12884 raft_consensus.cc:729] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7986054924fa40e5b7e3263a45183623, State: Initialized, Role: FOLLOWER
I20260928 16:36:42.029927 12892 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.030263 12892 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:42.030149 12884 consensus_queue.cc:260] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:42.030745 12885 ts_tablet_manager.cc:1428] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 16:36:42.030850 12884 ts_tablet_manager.cc:1428] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 16:36:42.030911 12891 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.031287 12891 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:42.031903 12883 raft_consensus.cc:374] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:42.032088 12883 raft_consensus.cc:729] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a277a99edb44af9aa3181e63a87068, State: Initialized, Role: FOLLOWER
I20260928 16:36:42.032429 12883 consensus_queue.cc:260] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:42.033054 12890 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.033437 12890 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:42.034775 12883 ts_tablet_manager.cc:1428] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260928 16:36:42.337512 12892 raft_consensus.cc:482] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:36:42.337836 12892 raft_consensus.cc:504] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:42.338722 12892 leader_election.cc:288] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7986054924fa40e5b7e3263a45183623 (127.10.247.193:35659), 33a277a99edb44af9aa3181e63a87068 (127.10.247.194:37431)
I20260928 16:36:42.339566 12671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9193653597eb4f84b3d6d070e8858d35" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a277a99edb44af9aa3181e63a87068" is_pre_election: true
I20260928 16:36:42.339601 12535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9193653597eb4f84b3d6d070e8858d35" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7986054924fa40e5b7e3263a45183623" is_pre_election: true
I20260928 16:36:42.339875 12671 raft_consensus.cc:2454] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 0.
I20260928 16:36:42.339901 12535 raft_consensus.cc:2454] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 0.
I20260928 16:36:42.340457 12743 leader_election.cc:302] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 33a277a99edb44af9aa3181e63a87068, 7986054924fa40e5b7e3263a45183623, f1bbd18e041f4252b03a4e09b9256a2a; no voters:
I20260928 16:36:42.340824 12892 raft_consensus.cc:2781] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:36:42.340963 12892 raft_consensus.cc:482] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:36:42.341056 12892 raft_consensus.cc:3037] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:42.343551 12892 raft_consensus.cc:504] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:42.344182 12892 leader_election.cc:288] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 election: Requested vote from peers 7986054924fa40e5b7e3263a45183623 (127.10.247.193:35659), 33a277a99edb44af9aa3181e63a87068 (127.10.247.194:37431)
I20260928 16:36:42.344838 12535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9193653597eb4f84b3d6d070e8858d35" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7986054924fa40e5b7e3263a45183623"
I20260928 16:36:42.344867 12671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9193653597eb4f84b3d6d070e8858d35" candidate_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a277a99edb44af9aa3181e63a87068"
I20260928 16:36:42.345095 12535 raft_consensus.cc:3037] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:42.345129 12671 raft_consensus.cc:3037] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:42.347555 12535 raft_consensus.cc:2454] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 1.
I20260928 16:36:42.347884 12671 raft_consensus.cc:2454] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bbd18e041f4252b03a4e09b9256a2a in term 1.
I20260928 16:36:42.348232 12743 leader_election.cc:302] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7986054924fa40e5b7e3263a45183623, f1bbd18e041f4252b03a4e09b9256a2a; no voters:
I20260928 16:36:42.348538 12892 raft_consensus.cc:2781] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:42.348732 12892 raft_consensus.cc:686] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 LEADER]: Becoming Leader. State: Replica: f1bbd18e041f4252b03a4e09b9256a2a, State: Running, Role: LEADER
I20260928 16:36:42.349128 12892 consensus_queue.cc:237] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } }
I20260928 16:36:42.350208 12924 consensus_queue.cc:1035] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:42.351049 12902 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: TxnStatusTablet state changed. Reason: New leader f1bbd18e041f4252b03a4e09b9256a2a. Latest consensus state: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.351411 12902 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: This TxnStatusTablet replica's current role is: LEADER
I20260928 16:36:42.352125 12926 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 16:36:42.352398 12926 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 16:36:42.353972 12373 catalog_manager.cc:5495] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bbd18e041f4252b03a4e09b9256a2a (127.10.247.195). New cstate: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } health_report { overall_health: HEALTHY } } }
I20260928 16:36:42.364967 12924 consensus_queue.cc:1035] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:42.394901 12919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:42.387934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:39967 (local address 127.0.0.1:34206)
0928 16:36:42.388261 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:42.388285 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:42.388486 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:42.389264 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:42.389268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:42.389279 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 16:36:42.389486 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:42.389499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.391417 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.391421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:42.392537 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:42.392552 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.393648 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.393653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:42.393812 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:42.393894 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 16:36:42.394395 (+ 501us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 16:36:42.394409 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 16:36:42.394467 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 16:36:42.394607 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260928 16:36:42.395107 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:42.388073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34206 (local address 127.10.247.195:39967)
0928 16:36:42.388864 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:42.388871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:42.388899 (+ 28us) server_negotiation.cc:407] Connection header received
0928 16:36:42.388999 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:42.389006 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:42.389108 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:42.389242 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:42.389662 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.391283 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.392708 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.393467 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.393603 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:42.394113 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:42.394305 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:42.394405 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:42.394510 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 16:36:42.394606 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":397,"threads_started":1}
I20260928 16:36:42.400539 12671 raft_consensus.cc:1261] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068 [term 1 FOLLOWER]: Refusing update from remote peer f1bbd18e041f4252b03a4e09b9256a2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:42.400667 12535 raft_consensus.cc:1261] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623 [term 1 FOLLOWER]: Refusing update from remote peer f1bbd18e041f4252b03a4e09b9256a2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:42.401326 12924 consensus_queue.cc:1035] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:42.401640 12902 consensus_queue.cc:1035] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:42.406118 12891 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: TxnStatusTablet state changed. Reason: New leader f1bbd18e041f4252b03a4e09b9256a2a. Latest consensus state: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.406423 12891 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:42.406818 12890 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: TxnStatusTablet state changed. Reason: New leader f1bbd18e041f4252b03a4e09b9256a2a. Latest consensus state: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.407120 12890 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:42.411751 12924 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.412305 12924 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: This TxnStatusTablet replica's current role is: LEADER
I20260928 16:36:42.412640 12924 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.412896 12924 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P f1bbd18e041f4252b03a4e09b9256a2a: This TxnStatusTablet replica's current role is: LEADER
I20260928 16:36:42.413533 12932 mvcc.cc:205] Tried to move back new op lower bound from 7334352496224227328 to 7334352496025133056. Current Snapshot: MvccSnapshot[applied={T|T < 7334352496224227328}]
I20260928 16:36:42.418753 12890 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.419044 12890 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P 33a277a99edb44af9aa3181e63a87068: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:42.422248 12891 tablet_replica.cc:439] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986054924fa40e5b7e3263a45183623" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35659 } } peers { permanent_uuid: "33a277a99edb44af9aa3181e63a87068" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 37431 } } peers { permanent_uuid: "f1bbd18e041f4252b03a4e09b9256a2a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39967 } } }
I20260928 16:36:42.422623 12891 tablet_replica.cc:442] T 9193653597eb4f84b3d6d070e8858d35 P 7986054924fa40e5b7e3263a45183623: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:42.439499 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:42.431843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:43105 (local address 127.10.247.195:39967)
0928 16:36:42.432093 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:42.432101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:42.432805 (+ 704us) server_negotiation.cc:407] Connection header received
0928 16:36:42.432995 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:42.433001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:42.433091 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:42.433199 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:42.434005 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.435245 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.437318 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.437995 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.438125 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:42.438621 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:42.438774 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:42.438865 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:42.439011 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 16:36:42.439102 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":61}
I20260928 16:36:42.439761 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:42.431765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:39967 (local address 127.10.247.195:43105)
0928 16:36:42.432695 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:42.432715 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:42.432860 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:42.433208 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:42.433211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:42.433221 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 16:36:42.433846 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:42.433859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.435406 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.435411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:42.437158 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:42.437172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.438204 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.438210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:42.438350 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:42.438427 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 16:36:42.438888 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 16:36:42.438901 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 16:36:42.438977 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 16:36:42.439105 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":47,"thread_start_us":574,"threads_started":1}
I20260928 16:36:42.509482 12371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:42.509768 12371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:42.513317 12371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d3c8083ebdb49a7ac42474cca0dc68c: Sending DeleteTablet for 3 replicas of tablet f5df537754a048dab3a892fa321532a7
I20260928 16:36:42.514550 12510 tablet_service.cc:1514] Processing DeleteTablet for tablet f5df537754a048dab3a892fa321532a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:42 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:34846
I20260928 16:36:42.514693 12650 tablet_service.cc:1514] Processing DeleteTablet for tablet f5df537754a048dab3a892fa321532a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:42 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:42474
I20260928 16:36:42.514772 12793 tablet_service.cc:1514] Processing DeleteTablet for tablet f5df537754a048dab3a892fa321532a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:42 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:58126
I20260928 16:36:42.524406 12947 tablet_replica.cc:330] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: stopping tablet replica
I20260928 16:36:42.524914 12947 raft_consensus.cc:2229] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:36:42.525635 12947 raft_consensus.cc:2258] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:42.526724 12946 tablet_replica.cc:330] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: stopping tablet replica
I20260928 16:36:42.527001 12946 raft_consensus.cc:2229] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:42.527348 12946 raft_consensus.cc:2258] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:42.527774 12945 tablet_replica.cc:330] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: stopping tablet replica
I20260928 16:36:42.528069 12945 raft_consensus.cc:2229] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:42.528471 12945 raft_consensus.cc:2258] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:42.529019 12947 ts_tablet_manager.cc:1905] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:42.529928 12945 ts_tablet_manager.cc:1905] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:42.537941 12946 ts_tablet_manager.cc:1905] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:42.538528 12945 ts_tablet_manager.cc:1918] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:36:42.538789 12945 log.cc:1196] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/f5df537754a048dab3a892fa321532a7
I20260928 16:36:42.539384 12945 ts_tablet_manager.cc:1939] T f5df537754a048dab3a892fa321532a7 P 7986054924fa40e5b7e3263a45183623: Deleting consensus metadata
I20260928 16:36:42.540998 12359 catalog_manager.cc:4841] TS 7986054924fa40e5b7e3263a45183623 (127.10.247.193:35659): tablet f5df537754a048dab3a892fa321532a7 (table test-table [id=a22821b72759428aaac61d3a6e177ff3]) successfully deleted
I20260928 16:36:42.544659 12946 ts_tablet_manager.cc:1918] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:36:42.544844 12946 log.cc:1196] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/f5df537754a048dab3a892fa321532a7
I20260928 16:36:42.545353 12946 ts_tablet_manager.cc:1939] T f5df537754a048dab3a892fa321532a7 P 33a277a99edb44af9aa3181e63a87068: Deleting consensus metadata
I20260928 16:36:42.546422 12947 ts_tablet_manager.cc:1918] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:36:42.546751 12947 log.cc:1196] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/f5df537754a048dab3a892fa321532a7
I20260928 16:36:42.546857 12357 catalog_manager.cc:4841] TS 33a277a99edb44af9aa3181e63a87068 (127.10.247.194:37431): tablet f5df537754a048dab3a892fa321532a7 (table test-table [id=a22821b72759428aaac61d3a6e177ff3]) successfully deleted
I20260928 16:36:42.547369 12947 ts_tablet_manager.cc:1939] T f5df537754a048dab3a892fa321532a7 P f1bbd18e041f4252b03a4e09b9256a2a: Deleting consensus metadata
I20260928 16:36:42.548632 12358 catalog_manager.cc:4841] TS f1bbd18e041f4252b03a4e09b9256a2a (127.10.247.195:39967): tablet f5df537754a048dab3a892fa321532a7 (table test-table [id=a22821b72759428aaac61d3a6e177ff3]) successfully deleted
Sep 28 16:36:42 dist-test-slave-9fn6 krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613401, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:36:42.566359 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:42.531504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59264 (local address 127.10.247.193:35659)
0928 16:36:42.546318 (+ 14814us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:42.546326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:42.546357 (+ 31us) server_negotiation.cc:407] Connection header received
0928 16:36:42.546451 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:42.546457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:42.546583 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:42.548462 (+ 1879us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:42.549542 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.550360 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.552340 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.552853 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.552955 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:42.558899 (+ 5944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:42.558921 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:42.558944 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:42.558998 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:42.562930 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.563995 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.564004 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.564007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.564215 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.564696 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.564701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.564704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.565123 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:42.565429 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:42.565788 (+ 359us) server_negotiation.cc:299] Negotiation successful
0928 16:36:42.566004 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14583,"thread_start_us":15234,"threads_started":1}
W20260928 16:36:42.567368 12575 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59264
I20260928 16:36:42.587772 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:42.570682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.10.247.254:42209)
0928 16:36:42.571637 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:42.571644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:42.571666 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:36:42.571738 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:42.571744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:42.571837 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:42.571968 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:42.573441 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.575100 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.576400 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.577194 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.577335 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:42.580075 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:42.580082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:42.580088 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:42.580144 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:42.584137 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.585093 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.585099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.585103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.585383 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.585814 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.585819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.585823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.586397 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:42.586705 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:42.587059 (+ 354us) server_negotiation.cc:299] Negotiation successful
0928 16:36:42.587332 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":618,"threads_started":1}
W20260928 16:36:42.588600 12371 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48780
I20260928 16:36:42.593804 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12444
I20260928 16:36:42.613520 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:42.594834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.10.247.193:35659)
0928 16:36:42.595086 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:42.595094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:42.597241 (+ 2147us) server_negotiation.cc:407] Connection header received
0928 16:36:42.597736 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:42.597744 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:42.597860 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:42.597945 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:42.599881 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.601246 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.603109 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.603851 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.604021 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:42.607351 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:42.607358 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:42.607363 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:42.607415 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:42.610694 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.611557 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.611561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.611563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.611716 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.612125 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.612128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.612130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.612423 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:42.612733 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:42.613010 (+ 277us) server_negotiation.cc:299] Negotiation successful
0928 16:36:42.613235 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260928 16:36:42.614203 12575 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59274
W20260928 16:36:42.614866 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12444: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:36:42.615118 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 12444
I20260928 16:36:42.640396 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12584
Sep 28 16:36:42 dist-test-slave-9fn6 krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613401, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:36:42.658831 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:42.641502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49860 (local address 127.10.247.194:37431)
0928 16:36:42.642148 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:42.642152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:42.642171 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:36:42.642241 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:42.642245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:42.642322 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:42.642429 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:42.643817 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.644814 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.646280 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.646793 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.646921 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:42.651947 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:42.651971 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:42.651992 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:42.652043 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:42.655307 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.656312 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.656323 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.656327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.656518 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.657053 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.657058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.657061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.657536 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:42.657832 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:42.658189 (+ 357us) server_negotiation.cc:299] Negotiation successful
0928 16:36:42.658430 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":311,"threads_started":1}
W20260928 16:36:42.659825 12716 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49860
W20260928 16:36:42.660648 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12584: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:36:42.660848 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 12584
I20260928 16:36:42.683187 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12725
Sep 28 16:36:42 dist-test-slave-9fn6 krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613401, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:36:42.702867 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:42.684199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.10.247.195:39967)
0928 16:36:42.684449 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:42.684456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:42.684485 (+ 29us) server_negotiation.cc:407] Connection header received
0928 16:36:42.684757 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:42.684764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:42.684877 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:42.685002 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:42.686371 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.687576 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.688936 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:42.689687 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:42.689832 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:42.694698 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:42.694726 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:42.694753 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:42.694825 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:42.699217 (+ 4392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.700131 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.700143 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.700147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.700355 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:42.700942 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:42.700949 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:42.700952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:42.701545 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:42.701919 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:42.702131 (+ 212us) server_negotiation.cc:299] Negotiation successful
0928 16:36:42.702427 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260928 16:36:42.703709 12858 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34214
W20260928 16:36:42.704504 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12725: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:36:42.704741 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 12725
I20260928 16:36:42.729163 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12341
I20260928 16:36:42.730659 12413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:42.862353 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 12341
2026-09-28T16:36:42Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5325 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:36:42 dist-test-slave-9fn6 krb5kdc[12959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:36:42 dist-test-slave-9fn6 krb5kdc[12959](info): set up 2 sockets
Sep 28 16:36:42 dist-test-slave-9fn6 krb5kdc[12959](info): commencing operation
krb5kdc: starting...
W20260928 16:36:43.889151 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:36:44 dist-test-slave-9fn6 krb5kdc[12959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:36:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:36:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:36:44.106549 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:46103
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:46103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:44.483721 12973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:44.484134 12973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:44.484234 12973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:44.492141 12973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:36:44.492295 12973 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 16:36:44.492342 12973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:44.492409 12973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:44.492462 12973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:36:44.492534 12973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:36:44.503518 12973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:46103
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:46103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:44.505193 12973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:44.506767 12973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:44.520409 12978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:44.520403 12979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:44.520543 12981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:44.521449 12973 server_base.cc:1034] running on GCE node
I20260928 16:36:44.522439 12973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:44.524338 12973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:44.525619 12973 hybrid_clock.cc:648] HybridClock initialized: now 1790613404525537 us; error 64 us; skew 500 ppm
I20260928 16:36:44.526192 12973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:44 dist-test-slave-9fn6 krb5kdc[12959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613404, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:44.530412 12973 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:36:44.532382 12973 webserver.cc:460] Webserver started at http://127.10.247.254:44887/ using document root <none> and password file <none>
I20260928 16:36:44.533354 12973 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:44.533478 12973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:44.533850 12973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:44.537779 12973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "34f8cd927c03423d8d570136eec1f4cd"
format_stamp: "Formatted at 2026-09-28 16:36:44 on dist-test-slave-9fn6"
server_key: "c77b42ee41a0ffea492b9f8235e885e5"
server_key_iv: "5929893d8dc500956deec071b31d3e52"
server_key_version: "encryptionkey@0"
I20260928 16:36:44.538754 12973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "34f8cd927c03423d8d570136eec1f4cd"
format_stamp: "Formatted at 2026-09-28 16:36:44 on dist-test-slave-9fn6"
server_key: "c77b42ee41a0ffea492b9f8235e885e5"
server_key_iv: "5929893d8dc500956deec071b31d3e52"
server_key_version: "encryptionkey@0"
I20260928 16:36:44.545768 12973 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260928 16:36:44.550415 12988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:44.552330 12973 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:36:44.552574 12973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "34f8cd927c03423d8d570136eec1f4cd"
format_stamp: "Formatted at 2026-09-28 16:36:44 on dist-test-slave-9fn6"
server_key: "c77b42ee41a0ffea492b9f8235e885e5"
server_key_iv: "5929893d8dc500956deec071b31d3e52"
server_key_version: "encryptionkey@0"
I20260928 16:36:44.552898 12973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:44.574826 12973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:44.580341 12973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:44.580688 12973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:44.604560 12973 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:46103
I20260928 16:36:44.604584 13049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:46103 every 8 connection(s)
I20260928 16:36:44.606312 12973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:36:44.610945 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 12973
I20260928 16:36:44.611192 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:36:44.611310 13050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:44.612118 11231 external_mini_cluster.cc:1428] Setting key ed5168c46b8ad5c06301b5a81fc2afcf
I20260928 16:36:44.629416 13050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd: Bootstrap starting.
Sep 28 16:36:44 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:44.633818 13050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:44.635164 13050 log.cc:824] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:44.639163 13050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd: No bootstrap required, opened a new log
I20260928 16:36:44.641498 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:44.617311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47608 (local address 127.10.247.254:46103)
0928 16:36:44.618309 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:44.618325 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:44.618382 (+ 57us) server_negotiation.cc:407] Connection header received
0928 16:36:44.620149 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:44.620187 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:44.620822 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:44.621327 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:44.623033 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:44.624509 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:44.625957 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:44.626832 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:44.627039 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:44.631995 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:44.632006 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:44.632025 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:44.632081 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:44.635463 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:44.636469 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:44.636488 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:44.636504 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:44.636799 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:44.637344 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:44.637348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:44.637350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:44.638035 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:44.638309 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:44.638810 (+ 501us) server_negotiation.cc:299] Negotiation successful
0928 16:36:44.639647 (+ 837us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":445,"threads_started":1}
I20260928 16:36:44.649063 13050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f8cd927c03423d8d570136eec1f4cd" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46103 } }
I20260928 16:36:44.649654 13050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:44.649780 13050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34f8cd927c03423d8d570136eec1f4cd, State: Initialized, Role: FOLLOWER
I20260928 16:36:44.650506 13050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f8cd927c03423d8d570136eec1f4cd" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46103 } }
I20260928 16:36:44.650753 13050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:36:44.650866 13050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:36:44.651021 13050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:44.653712 13050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f8cd927c03423d8d570136eec1f4cd" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46103 } }
I20260928 16:36:44.654309 13050 leader_election.cc:302] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34f8cd927c03423d8d570136eec1f4cd; no voters:
I20260928 16:36:44.654938 13050 leader_election.cc:288] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:36:44.655081 13055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:44.656395 13055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [term 1 LEADER]: Becoming Leader. State: Replica: 34f8cd927c03423d8d570136eec1f4cd, State: Running, Role: LEADER
I20260928 16:36:44.657142 13055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f8cd927c03423d8d570136eec1f4cd" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46103 } }
I20260928 16:36:44.657639 13050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:36:44.660637 13057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34f8cd927c03423d8d570136eec1f4cd. Latest consensus state: current_term: 1 leader_uuid: "34f8cd927c03423d8d570136eec1f4cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f8cd927c03423d8d570136eec1f4cd" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46103 } } }
I20260928 16:36:44.660817 13056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34f8cd927c03423d8d570136eec1f4cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f8cd927c03423d8d570136eec1f4cd" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46103 } } }
I20260928 16:36:44.661108 13057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:44.661113 13056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:44.662075 13065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:36:44.668840 13065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:36:44.679090 13065 catalog_manager.cc:1347] Generated new cluster ID: 4a3bc60adfe848d89d389674b69fe4c2
I20260928 16:36:44.679280 13065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:36:44.696012 13065 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:36:44.698269 13065 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:36:44.724712 13065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd: Generated new TSK 0
I20260928 16:36:44.726250 13065 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:36:44.794579 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:46103
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:45.181042 13076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:45.181523 13076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:45.181702 13076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:45.190231 13076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:45.190424 13076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:45.190565 13076 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 16:36:45.190640 13076 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 16:36:45.190752 13076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:36:45.202503 13076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.247.254:46103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:45.204413 13076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:45.206284 13076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:45.222488 13081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:45.222460 13084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:45.222582 13082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:45.223357 13076 server_base.cc:1034] running on GCE node
I20260928 16:36:45.224243 13076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:45.225664 13076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:45.226974 13076 hybrid_clock.cc:648] HybridClock initialized: now 1790613405226838 us; error 67 us; skew 500 ppm
I20260928 16:36:45.227406 13076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:45 dist-test-slave-9fn6 krb5kdc[12959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613405, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:45.237843 13076 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:36:45.240082 13076 webserver.cc:460] Webserver started at http://127.10.247.193:33877/ using document root <none> and password file <none>
I20260928 16:36:45.241163 13076 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:45.241330 13076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:45.241740 13076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:45.245752 13076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "bf41783244904579a96610aa89263e42"
format_stamp: "Formatted at 2026-09-28 16:36:45 on dist-test-slave-9fn6"
server_key: "ca6f21e62fb85abbb8211973914e658d"
server_key_iv: "f58aabed7adbd71cd8b1eeec068ede28"
server_key_version: "encryptionkey@0"
I20260928 16:36:45.246797 13076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "bf41783244904579a96610aa89263e42"
format_stamp: "Formatted at 2026-09-28 16:36:45 on dist-test-slave-9fn6"
server_key: "ca6f21e62fb85abbb8211973914e658d"
server_key_iv: "f58aabed7adbd71cd8b1eeec068ede28"
server_key_version: "encryptionkey@0"
I20260928 16:36:45.253695 13076 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260928 16:36:45.258356 13091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:45.260239 13076 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:36:45.260440 13076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "bf41783244904579a96610aa89263e42"
format_stamp: "Formatted at 2026-09-28 16:36:45 on dist-test-slave-9fn6"
server_key: "ca6f21e62fb85abbb8211973914e658d"
server_key_iv: "f58aabed7adbd71cd8b1eeec068ede28"
server_key_version: "encryptionkey@0"
I20260928 16:36:45.260736 13076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:45.290647 13076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:45.295471 13076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:45.295826 13076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:45.299182 13076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:45.299315 13076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:45.299448 13076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:45.299526 13076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:36:45 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613405, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:45.326982 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:45.300206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:55123 (local address 127.10.247.254:46103)
0928 16:36:45.300848 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:45.300857 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:45.302371 (+ 1514us) server_negotiation.cc:407] Connection header received
0928 16:36:45.305834 (+ 3463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:45.305842 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:45.305970 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:45.306087 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:45.309366 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.310523 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.312491 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.313091 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.313245 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:45.319507 (+ 6262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:45.319515 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:45.319521 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:45.319581 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:45.322751 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:45.323950 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:45.323954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:45.323956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:45.324097 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:45.324715 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:45.324718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:45.324721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:45.325035 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:45.325163 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:45.326424 (+ 1261us) server_negotiation.cc:299] Negotiation successful
0928 16:36:45.326672 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":300,"threads_started":1}
I20260928 16:36:45.329320 13102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:45.301022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46103 (local address 127.10.247.193:55123)
0928 16:36:45.302099 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:45.302170 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:45.305323 (+ 3153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:45.306504 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:45.306520 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:45.307707 (+ 1187us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:45.309042 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:45.309088 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.310746 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.310752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:45.312312 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:45.312328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.313436 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.313442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:45.313869 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:45.314912 (+ 1043us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:45.314954 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:45.319105 (+ 4151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:45.322937 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:45.322951 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:45.322982 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:45.323815 (+ 833us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:45.324196 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:45.324201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:45.324206 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:45.324561 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:45.325164 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:45.325176 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:45.325897 (+ 721us) client_negotiation.cc:769] Sending connection context
0928 16:36:45.326292 (+ 395us) client_negotiation.cc:240] Negotiation successful
0928 16:36:45.327089 (+ 797us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":399,"threads_started":1}
I20260928 16:36:45.354549 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:45.337144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:47883 (local address 127.10.247.254:46103)
0928 16:36:45.337369 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:45.337376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:45.338169 (+ 793us) server_negotiation.cc:407] Connection header received
0928 16:36:45.338504 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:45.338511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:45.338632 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:45.338763 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:45.340664 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.341746 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.343821 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.344279 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.344376 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:45.347163 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:45.347172 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:45.347178 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:45.347230 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:45.350756 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:45.351857 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:45.351864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:45.351867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:45.352082 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:45.352523 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:45.352527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:45.352529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:45.352929 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:45.353058 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:45.353833 (+ 775us) server_negotiation.cc:299] Negotiation successful
0928 16:36:45.354135 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:36:45.354542 13076 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:34131
I20260928 16:36:45.354606 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:45.337219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46103 (local address 127.10.247.193:47883)
0928 16:36:45.338032 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:45.338056 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:45.338254 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:45.338818 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:45.338824 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:45.339794 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:45.340424 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:45.340444 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.341889 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.341896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:45.343678 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:45.343696 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.344467 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.344474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:45.344706 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:45.345538 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:45.345562 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:45.346879 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:45.350912 (+ 4033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:45.350922 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:45.350926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:45.351672 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:45.352224 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:45.352229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:45.352233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:45.352402 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:45.353053 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:45.353059 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:45.353333 (+ 274us) client_negotiation.cc:769] Sending connection context
0928 16:36:45.353527 (+ 194us) client_negotiation.cc:240] Negotiation successful
0928 16:36:45.354021 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":412,"threads_started":1}
I20260928 16:36:45.354799 13212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:34131 every 8 connection(s)
I20260928 16:36:45.356513 13076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:36:45.359040 13213 heartbeater.cc:347] Connected to a master server at 127.10.247.254:46103
I20260928 16:36:45.359354 13213 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:45.360255 13213 heartbeater.cc:511] Master 127.10.247.254:46103 requested a full tablet report, sending...
I20260928 16:36:45.362427 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 13076
I20260928 16:36:45.362524 13005 ts_manager.cc:194] Registered new tserver with Master: bf41783244904579a96610aa89263e42 (127.10.247.193:34131)
I20260928 16:36:45.362643 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:36:45.363345 11231 external_mini_cluster.cc:1428] Setting key e0450bcc05927091920b3359bb644fa7
I20260928 16:36:45.364404 13005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:55123
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:36:45.412423 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:46103
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:45.810371 13217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:45.810784 13217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:45.810936 13217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:45.819145 13217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:45.819331 13217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:45.819483 13217 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 16:36:45.819561 13217 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 16:36:45.819679 13217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:36:45.832278 13217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.247.254:46103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:45.834131 13217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:45.835924 13217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:45.851220 13222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:45.851240 13223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:45.851756 13225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:45.852730 13217 server_base.cc:1034] running on GCE node
I20260928 16:36:45.853467 13217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:45.854607 13217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:45.855825 13217 hybrid_clock.cc:648] HybridClock initialized: now 1790613405855765 us; error 53 us; skew 500 ppm
I20260928 16:36:45.856221 13217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:45 dist-test-slave-9fn6 krb5kdc[12959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613405, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:45.860203 13217 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:36:45.862200 13217 webserver.cc:460] Webserver started at http://127.10.247.194:35501/ using document root <none> and password file <none>
I20260928 16:36:45.863099 13217 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:45.863240 13217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:45.863629 13217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:45.867489 13217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "066d7245e5a946aeb7b0c20b62816dd4"
format_stamp: "Formatted at 2026-09-28 16:36:45 on dist-test-slave-9fn6"
server_key: "22316c84c7bef23ce930aecdce17f101"
server_key_iv: "d0e795a18fd0f54b9cefbcd6e29bf839"
server_key_version: "encryptionkey@0"
I20260928 16:36:45.868469 13217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "066d7245e5a946aeb7b0c20b62816dd4"
format_stamp: "Formatted at 2026-09-28 16:36:45 on dist-test-slave-9fn6"
server_key: "22316c84c7bef23ce930aecdce17f101"
server_key_iv: "d0e795a18fd0f54b9cefbcd6e29bf839"
server_key_version: "encryptionkey@0"
I20260928 16:36:45.874951 13217 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:36:45.879460 13232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:45.881424 13217 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:36:45.881646 13217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "066d7245e5a946aeb7b0c20b62816dd4"
format_stamp: "Formatted at 2026-09-28 16:36:45 on dist-test-slave-9fn6"
server_key: "22316c84c7bef23ce930aecdce17f101"
server_key_iv: "d0e795a18fd0f54b9cefbcd6e29bf839"
server_key_version: "encryptionkey@0"
I20260928 16:36:45.881899 13217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:45.912951 13217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:45.917832 13217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:45.918174 13217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:45.921603 13217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:45.921751 13217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:45.921893 13217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:45.921994 13217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:36:45 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613405, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:45.952436 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:45.922272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:34307 (local address 127.10.247.254:46103)
0928 16:36:45.923078 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:45.923084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:45.924396 (+ 1312us) server_negotiation.cc:407] Connection header received
0928 16:36:45.927142 (+ 2746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:45.927151 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:45.927274 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:45.927417 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:45.930629 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.931994 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.936329 (+ 4335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.937128 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.937264 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:45.944008 (+ 6744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:45.944015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:45.944019 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:45.944073 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:45.947557 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:45.948776 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:45.948782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:45.948787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:45.948979 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:45.949780 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:45.949788 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:45.949791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:45.950209 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:45.950391 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:45.951681 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0928 16:36:45.951978 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":561,"threads_started":1}
I20260928 16:36:45.953883 13242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:45.923219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46103 (local address 127.10.247.194:34307)
0928 16:36:45.924186 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:45.924238 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:45.926699 (+ 2461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:45.927778 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:45.927791 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:45.928890 (+ 1099us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:45.930302 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:45.930335 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.932246 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.932253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:45.936003 (+ 3750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:45.936032 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.937748 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.937757 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:45.938323 (+ 566us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:45.939579 (+ 1256us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:45.939628 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:45.943549 (+ 3921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:45.947760 (+ 4211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:45.947775 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:45.947806 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:45.948583 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:45.949162 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:45.949168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:45.949173 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:45.949602 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:45.950507 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:45.950519 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:45.951222 (+ 703us) client_negotiation.cc:769] Sending connection context
0928 16:36:45.951552 (+ 330us) client_negotiation.cc:240] Negotiation successful
0928 16:36:45.952283 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":392,"threads_started":1}
I20260928 16:36:45.974645 13217 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:38205
I20260928 16:36:45.974680 13353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:38205 every 8 connection(s)
I20260928 16:36:45.976459 13217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:36:45.978281 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 13217
I20260928 16:36:45.978513 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:36:45.979228 11231 external_mini_cluster.cc:1428] Setting key 081b46aeed94d816c31a84e7e43ddb2b
I20260928 16:36:45.979611 13354 heartbeater.cc:347] Connected to a master server at 127.10.247.254:46103
I20260928 16:36:45.979946 13354 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:45.981154 13354 heartbeater.cc:511] Master 127.10.247.254:46103 requested a full tablet report, sending...
I20260928 16:36:45.982271 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:45.962762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:35315 (local address 127.10.247.254:46103)
0928 16:36:45.962986 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:45.962992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:45.964144 (+ 1152us) server_negotiation.cc:407] Connection header received
0928 16:36:45.964403 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:45.964416 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:45.964521 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:45.964657 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:45.965972 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.967292 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.969793 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.970448 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.970566 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:45.973442 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:45.973449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:45.973454 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:45.973502 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:45.977369 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:45.978584 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:45.978592 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:45.978596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:45.978799 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:45.979293 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:45.979299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:45.979303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:45.979958 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:45.980260 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:45.981529 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0928 16:36:45.981848 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 16:36:45.983639 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:45.962888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46103 (local address 127.10.247.194:35315)
0928 16:36:45.964019 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:45.964044 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:45.964250 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:45.964747 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:45.964751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:45.965293 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:45.965803 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:45.965820 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.967518 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.967524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:45.969607 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:45.969627 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:45.970689 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:45.970697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:45.970904 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:45.971935 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:45.971964 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:45.973198 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:45.977579 (+ 4381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:45.977586 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:45.977591 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:45.978375 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:45.978961 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:45.978967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:45.978972 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:45.979152 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:45.980334 (+ 1182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:45.980340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:45.980663 (+ 323us) client_negotiation.cc:769] Sending connection context
0928 16:36:45.980832 (+ 169us) client_negotiation.cc:240] Negotiation successful
0928 16:36:45.983010 (+ 2178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"thread_start_us":768,"threads_started":1}
I20260928 16:36:45.985149 13004 ts_manager.cc:194] Registered new tserver with Master: 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205)
I20260928 16:36:45.986526 13004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:34307
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:36:46.032662 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:46103
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:36:46.367990 13213 heartbeater.cc:503] Master 127.10.247.254:46103 was elected leader, sending a full tablet report...
W20260928 16:36:46.419548 13358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:46.419881 13358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:46.420045 13358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:46.427785 13358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:46.427908 13358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:46.428032 13358 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 16:36:46.428108 13358 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 16:36:46.428210 13358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:36:46.439272 13358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.247.254:46103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:46.441133 13358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:46.442924 13358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:46.458463 13363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:46.458504 13364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:46.458586 13366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:46.459726 13358 server_base.cc:1034] running on GCE node
I20260928 16:36:46.460464 13358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:46.461746 13358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:46.462958 13358 hybrid_clock.cc:648] HybridClock initialized: now 1790613406462896 us; error 66 us; skew 500 ppm
I20260928 16:36:46.463310 13358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:46 dist-test-slave-9fn6 krb5kdc[12959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613406, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:46.466883 13358 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:36:46.468775 13358 webserver.cc:460] Webserver started at http://127.10.247.195:46163/ using document root <none> and password file <none>
I20260928 16:36:46.469623 13358 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:46.469758 13358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:46.470113 13358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:46.473605 13358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "a75bf71c13314e26b2c2e5f1f7738d8a"
format_stamp: "Formatted at 2026-09-28 16:36:46 on dist-test-slave-9fn6"
server_key: "7808082cca6d2eb577c127104498b58e"
server_key_iv: "ce2345bcfbac0377ed58ad7f35368ede"
server_key_version: "encryptionkey@0"
I20260928 16:36:46.474471 13358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "a75bf71c13314e26b2c2e5f1f7738d8a"
format_stamp: "Formatted at 2026-09-28 16:36:46 on dist-test-slave-9fn6"
server_key: "7808082cca6d2eb577c127104498b58e"
server_key_iv: "ce2345bcfbac0377ed58ad7f35368ede"
server_key_version: "encryptionkey@0"
I20260928 16:36:46.480717 13358 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:36:46.484815 13373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:46.486389 13358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:36:46.486538 13358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "a75bf71c13314e26b2c2e5f1f7738d8a"
format_stamp: "Formatted at 2026-09-28 16:36:46 on dist-test-slave-9fn6"
server_key: "7808082cca6d2eb577c127104498b58e"
server_key_iv: "ce2345bcfbac0377ed58ad7f35368ede"
server_key_version: "encryptionkey@0"
I20260928 16:36:46.486729 13358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:46.522578 13358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:46.527612 13358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:46.527918 13358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:46.531097 13358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:46.531224 13358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:46.531363 13358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:46.531448 13358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:36:46 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613406, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:46.558658 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.532219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:53479 (local address 127.10.247.254:46103)
0928 16:36:46.532743 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:46.532751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:46.534185 (+ 1434us) server_negotiation.cc:407] Connection header received
0928 16:36:46.536478 (+ 2293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:46.536485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:46.536586 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:46.536775 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:46.539539 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.540655 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.543834 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.544452 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.544643 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.550185 (+ 5542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:46.550194 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:46.550201 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:46.550268 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:46.554059 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:46.554944 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:46.554952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:46.554956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:46.555154 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:46.555844 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:46.555850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:46.555854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:46.556369 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:46.556572 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:46.557860 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0928 16:36:46.558168 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":242,"threads_started":1}
I20260928 16:36:46.560159 13383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.533061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46103 (local address 127.10.247.195:53479)
0928 16:36:46.533978 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:46.534023 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:46.536142 (+ 2119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:46.537267 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:46.537284 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:46.538177 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:46.539307 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.539340 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.540867 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.540873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.543610 (+ 2737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.543634 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.544778 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.544790 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.545066 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.546094 (+ 1028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:46.546136 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:46.549821 (+ 3685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:46.554239 (+ 4418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:46.554251 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:46.554275 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:46.554805 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:46.555320 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:46.555325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:46.555330 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:46.555668 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:46.556547 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:46.556559 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:46.557330 (+ 771us) client_negotiation.cc:769] Sending connection context
0928 16:36:46.557756 (+ 426us) client_negotiation.cc:240] Negotiation successful
0928 16:36:46.558436 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":391,"threads_started":1}
I20260928 16:36:46.583343 13494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:39607 every 8 connection(s)
I20260928 16:36:46.583351 13358 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:39607
I20260928 16:36:46.585371 13358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:36:46.588146 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 13358
I20260928 16:36:46.588351 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:36:46.588362 13495 heartbeater.cc:347] Connected to a master server at 127.10.247.254:46103
I20260928 16:36:46.588649 13495 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:46.588979 11231 external_mini_cluster.cc:1428] Setting key 52222206e047049f5deb0d3a6eb29fa4
I20260928 16:36:46.589591 13495 heartbeater.cc:511] Master 127.10.247.254:46103 requested a full tablet report, sending...
I20260928 16:36:46.589866 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.573044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:51607 (local address 127.10.247.254:46103)
0928 16:36:46.573270 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:46.573278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:46.574198 (+ 920us) server_negotiation.cc:407] Connection header received
0928 16:36:46.574416 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:46.574422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:46.574543 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:46.574665 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:46.575866 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.577151 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.579371 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.580118 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.580256 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.583007 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:46.583015 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:46.583020 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:46.583067 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:46.586551 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:46.587391 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:46.587397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:46.587402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:46.587575 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:46.588072 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:46.588076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:46.588079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:46.588498 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:46.588689 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:46.589230 (+ 541us) server_negotiation.cc:299] Negotiation successful
0928 16:36:46.589503 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:36:46.590003 13470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.573427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46103 (local address 127.10.247.195:51607)
0928 16:36:46.574068 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:46.574086 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:46.574268 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:46.574688 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:46.574691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:46.575348 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:46.575694 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.575707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.577308 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.577314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.579191 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.579216 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.580549 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.580556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.580773 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.581544 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:46.581562 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:46.582754 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:46.586685 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:46.586691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:46.586695 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:46.587281 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:46.587737 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:46.587741 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:46.587744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:46.587906 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:46.588688 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:46.588694 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:46.588920 (+ 226us) client_negotiation.cc:769] Sending connection context
0928 16:36:46.589115 (+ 195us) client_negotiation.cc:240] Negotiation successful
0928 16:36:46.589517 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":335,"threads_started":1}
I20260928 16:36:46.591408 13005 ts_manager.cc:194] Registered new tserver with Master: a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607)
I20260928 16:36:46.592645 13005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:53479
I20260928 16:36:46.593080 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 16:36:46 dist-test-slave-9fn6 krb5kdc[12959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 16:36:46 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:46.649286 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.633448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.10.247.254:46103)
0928 16:36:46.633749 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:46.633758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:46.634037 (+ 279us) server_negotiation.cc:407] Connection header received
0928 16:36:46.634246 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:46.634251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:46.634352 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:46.634442 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:46.635969 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.636900 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.638227 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.638671 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.638767 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.643420 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:46.643426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:46.643430 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:46.643482 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:46.646358 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:46.647225 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:46.647230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:46.647232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:46.647374 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:46.647742 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:46.647746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:46.647748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:46.648074 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:46.648304 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:46.648566 (+ 262us) server_negotiation.cc:299] Negotiation successful
0928 16:36:46.648890 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260928 16:36:46.654593 13005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:36:46.663020 13005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:36:46.692508 13508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.678087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.10.247.193:34131)
0928 16:36:46.678932 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:46.678945 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:46.680438 (+ 1493us) server_negotiation.cc:407] Connection header received
0928 16:36:46.680574 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:46.680583 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:46.680936 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:46.681142 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:46.682624 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.684406 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.687771 (+ 3365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.689630 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.689832 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.691807 (+ 1975us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:46.691917 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 16:36:46.692158 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":491,"threads_started":1}
I20260928 16:36:46.691701 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.677868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:34131 (local address 127.0.0.1:45614)
0928 16:36:46.679921 (+ 2053us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:46.679977 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:46.680222 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:46.681758 (+ 1536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:46.681768 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:46.681808 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:46.682328 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.682343 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.684654 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.684661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.687508 (+ 2847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.687528 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.690318 (+ 2790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.690326 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.690594 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.690702 (+ 108us) client_negotiation.cc:769] Sending connection context
0928 16:36:46.690865 (+ 163us) client_negotiation.cc:240] Negotiation successful
0928 16:36:46.691155 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1717,"threads_started":1}
I20260928 16:36:46.700086 13143 tablet_service.cc:1467] Processing CreateTablet for tablet 96b7738b758f4cf58fb8bd4f8c8c765e (DEFAULT_TABLE table=test-table [id=8dff93ea5d434fc8b975b69103d47bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:46.701884 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.678813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:38205 (local address 127.0.0.1:55442)
0928 16:36:46.692996 (+ 14183us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:46.693018 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:46.693133 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:46.695715 (+ 2582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:46.695721 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:46.695748 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:46.695952 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.695965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.698039 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.698044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.699262 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.699274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.701144 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.701150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.701332 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.701371 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 16:36:46.701442 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 16:36:46.701598 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14042,"thread_start_us":600,"threads_started":1}
I20260928 16:36:46.702325 13143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96b7738b758f4cf58fb8bd4f8c8c765e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:46.702821 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.678408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55442 (local address 127.10.247.194:38205)
0928 16:36:46.694855 (+ 16447us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:46.694868 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:46.694912 (+ 44us) server_negotiation.cc:407] Connection header received
0928 16:36:46.695052 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:46.695062 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:46.695443 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:46.696151 (+ 708us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:46.696444 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.697907 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.699414 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.700952 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.702187 (+ 1235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.702359 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:46.702441 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 16:36:46.702597 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16256,"thread_start_us":16104,"threads_started":1}
I20260928 16:36:46.704139 13509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.680349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:39607 (local address 127.0.0.1:36582)
0928 16:36:46.693871 (+ 13522us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:46.693894 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:46.694021 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:46.696994 (+ 2973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:46.696999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:46.697025 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:46.697430 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.697442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.699509 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.699514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.701581 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.701592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.703368 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.703375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.703557 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.703590 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:36:46.703673 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 16:36:46.703813 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13372,"thread_start_us":16291,"threads_started":1}
I20260928 16:36:46.704312 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.681079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36582 (local address 127.10.247.195:39607)
0928 16:36:46.695426 (+ 14347us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:46.695434 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:46.695468 (+ 34us) server_negotiation.cc:407] Connection header received
0928 16:36:46.695570 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:46.695580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:46.695814 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:46.696155 (+ 341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:46.697671 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.699379 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.701754 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.703169 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.703365 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.703533 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:46.703752 (+ 219us) server_negotiation.cc:299] Negotiation successful
0928 16:36:46.703920 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14144,"thread_start_us":13939,"threads_started":1}
I20260928 16:36:46.706909 13284 tablet_service.cc:1467] Processing CreateTablet for tablet 96b7738b758f4cf58fb8bd4f8c8c765e (DEFAULT_TABLE table=test-table [id=8dff93ea5d434fc8b975b69103d47bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:46.707163 13425 tablet_service.cc:1467] Processing CreateTablet for tablet 96b7738b758f4cf58fb8bd4f8c8c765e (DEFAULT_TABLE table=test-table [id=8dff93ea5d434fc8b975b69103d47bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:46.708742 13284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96b7738b758f4cf58fb8bd4f8c8c765e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:46.708742 13425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96b7738b758f4cf58fb8bd4f8c8c765e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:46.721227 13514 tablet_bootstrap.cc:492] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: Bootstrap starting.
I20260928 16:36:46.723852 13516 tablet_bootstrap.cc:492] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: Bootstrap starting.
I20260928 16:36:46.725546 13515 tablet_bootstrap.cc:492] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: Bootstrap starting.
I20260928 16:36:46.725911 13514 tablet_bootstrap.cc:654] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:46.727519 13516 tablet_bootstrap.cc:654] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:46.727573 13514 log.cc:824] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:46.728801 13515 tablet_bootstrap.cc:654] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:46.729029 13516 log.cc:824] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:46.730175 13515 log.cc:824] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:46.732466 13514 tablet_bootstrap.cc:492] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: No bootstrap required, opened a new log
I20260928 16:36:46.732527 13516 tablet_bootstrap.cc:492] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: No bootstrap required, opened a new log
I20260928 16:36:46.732795 13514 ts_tablet_manager.cc:1397] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260928 16:36:46.732877 13516 ts_tablet_manager.cc:1397] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260928 16:36:46.733459 13515 tablet_bootstrap.cc:492] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: No bootstrap required, opened a new log
I20260928 16:36:46.733735 13515 ts_tablet_manager.cc:1397] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260928 16:36:46.740592 13514 raft_consensus.cc:348] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:46.741063 13514 raft_consensus.cc:374] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:46.741187 13514 raft_consensus.cc:729] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75bf71c13314e26b2c2e5f1f7738d8a, State: Initialized, Role: FOLLOWER
I20260928 16:36:46.742008 13514 consensus_queue.cc:260] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:46.743822 13495 heartbeater.cc:503] Master 127.10.247.254:46103 was elected leader, sending a full tablet report...
I20260928 16:36:46.743994 13516 raft_consensus.cc:348] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:46.744148 13515 raft_consensus.cc:348] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:46.744491 13516 raft_consensus.cc:374] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:46.744549 13515 raft_consensus.cc:374] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:46.744593 13516 raft_consensus.cc:729] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066d7245e5a946aeb7b0c20b62816dd4, State: Initialized, Role: FOLLOWER
I20260928 16:36:46.744654 13515 raft_consensus.cc:729] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf41783244904579a96610aa89263e42, State: Initialized, Role: FOLLOWER
I20260928 16:36:46.744678 13514 ts_tablet_manager.cc:1428] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent starting tablet: real 0.012s user 0.005s sys 0.005s
I20260928 16:36:46.745285 13516 consensus_queue.cc:260] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:46.745316 13515 consensus_queue.cc:260] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:46.746975 13354 heartbeater.cc:503] Master 127.10.247.254:46103 was elected leader, sending a full tablet report...
I20260928 16:36:46.747565 13516 ts_tablet_manager.cc:1428] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260928 16:36:46.748332 13515 ts_tablet_manager.cc:1428] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260928 16:36:46.837513 13496 tablet.cc:2369] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:36:46.859853 13214 tablet.cc:2369] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:36:46.878240 13522 raft_consensus.cc:482] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:36:46.878633 13522 raft_consensus.cc:504] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:46.880769 13522 leader_election.cc:288] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205), a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607)
I20260928 16:36:46.888659 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.881156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:48309 (local address 127.10.247.194:38205)
0928 16:36:46.881440 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:46.881447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:46.882040 (+ 593us) server_negotiation.cc:407] Connection header received
0928 16:36:46.882127 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:46.882131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:46.882201 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:46.882288 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:46.883097 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.884156 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.886469 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.887484 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.887623 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.887687 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:46.888162 (+ 475us) server_negotiation.cc:299] Negotiation successful
0928 16:36:46.888318 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 16:36:46.888718 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.881035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:38205 (local address 127.10.247.193:48309)
0928 16:36:46.881681 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:46.881709 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:46.881919 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:46.882503 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:46.882507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:46.882534 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:46.882935 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.882949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.884291 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.884297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.886265 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.886304 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.887708 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.887716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.887938 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.887987 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 16:36:46.888113 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 16:36:46.888306 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":317,"threads_started":1}
I20260928 16:36:46.889110 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.881746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:39607 (local address 127.10.247.193:49565)
0928 16:36:46.882777 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:46.882794 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:46.882922 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:46.883265 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:46.883269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:46.883293 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:46.883702 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.883715 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.885024 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.885029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.886647 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.886662 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.888259 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.888265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.888467 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.888502 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 16:36:46.888596 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 16:36:46.888776 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"thread_start_us":700,"threads_started":1}
I20260928 16:36:46.889128 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.882184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:49565 (local address 127.10.247.195:39607)
0928 16:36:46.882439 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:46.882446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:46.882894 (+ 448us) server_negotiation.cc:407] Connection header received
0928 16:36:46.883059 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:46.883065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:46.883144 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:46.883216 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:46.883865 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.884885 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.886846 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.888017 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.888188 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.888275 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:46.888681 (+ 406us) server_negotiation.cc:299] Negotiation successful
0928 16:36:46.888837 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260928 16:36:46.890230 13304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96b7738b758f4cf58fb8bd4f8c8c765e" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4" is_pre_election: true
I20260928 16:36:46.890468 13445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96b7738b758f4cf58fb8bd4f8c8c765e" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" is_pre_election: true
I20260928 16:36:46.891013 13304 raft_consensus.cc:2454] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 0.
I20260928 16:36:46.891608 13445 raft_consensus.cc:2454] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 0.
I20260928 16:36:46.891968 13095 leader_election.cc:302] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066d7245e5a946aeb7b0c20b62816dd4, bf41783244904579a96610aa89263e42; no voters:
I20260928 16:36:46.892519 13522 raft_consensus.cc:2781] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:36:46.892746 13522 raft_consensus.cc:482] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:36:46.892869 13522 raft_consensus.cc:3037] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:46.895454 13522 raft_consensus.cc:504] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:46.896121 13522 leader_election.cc:288] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 election: Requested vote from peers 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205), a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607)
I20260928 16:36:46.896849 13304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96b7738b758f4cf58fb8bd4f8c8c765e" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4"
I20260928 16:36:46.896925 13445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96b7738b758f4cf58fb8bd4f8c8c765e" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a"
I20260928 16:36:46.897102 13304 raft_consensus.cc:3037] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:46.897161 13445 raft_consensus.cc:3037] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:46.899662 13304 raft_consensus.cc:2454] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 1.
I20260928 16:36:46.899680 13445 raft_consensus.cc:2454] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 1.
I20260928 16:36:46.900148 13095 leader_election.cc:302] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066d7245e5a946aeb7b0c20b62816dd4, bf41783244904579a96610aa89263e42; no voters:
I20260928 16:36:46.900496 13522 raft_consensus.cc:2781] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:46.901046 13522 raft_consensus.cc:686] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 1 LEADER]: Becoming Leader. State: Replica: bf41783244904579a96610aa89263e42, State: Running, Role: LEADER
I20260928 16:36:46.901613 13522 consensus_queue.cc:237] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:46.907577 13005 catalog_manager.cc:5495] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf41783244904579a96610aa89263e42 (127.10.247.193). New cstate: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } health_report { overall_health: UNKNOWN } } }
Sep 28 16:36:46 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613404, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:46.975626 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.960032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46103 (local address 127.0.0.1:47620)
0928 16:36:46.960817 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:46.960839 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:46.961074 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:46.961443 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:46.961449 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:46.962414 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:46.962810 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.962825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.964132 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.964136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.965341 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:46.965357 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.966294 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.966301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:46.966473 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.967208 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:46.967228 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:46.969810 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:46.972822 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:46.972840 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:46.972850 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:46.973348 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:46.973734 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:46.973738 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:46.973742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:46.973888 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:46.974449 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:46.974458 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:46.974650 (+ 192us) client_negotiation.cc:769] Sending connection context
0928 16:36:46.974793 (+ 143us) client_negotiation.cc:240] Negotiation successful
0928 16:36:46.975125 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":388,"threads_started":1}
I20260928 16:36:46.975625 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:46.960031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47620 (local address 127.10.247.254:46103)
0928 16:36:46.960244 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:46.960249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:46.960976 (+ 727us) server_negotiation.cc:407] Connection header received
0928 16:36:46.961211 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:46.961217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:46.961296 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:46.961404 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:46.963001 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.963974 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.965481 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:46.966073 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:46.966205 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:46.970036 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:46.970041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:46.970045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:46.970084 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:46.972675 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:46.973475 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:46.973480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:46.973482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:46.973627 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:46.973999 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:46.974003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:46.974005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:46.974320 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:46.974464 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:46.974912 (+ 448us) server_negotiation.cc:299] Negotiation successful
0928 16:36:46.975261 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":61}
I20260928 16:36:46.982370 13005 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:47620:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 16:36:46.983695 13005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260928 16:36:46.984773 13355 tablet.cc:2369] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:36:46.992846 13143 tablet_service.cc:1467] Processing CreateTablet for tablet 03e5cea41ef34868b96f5ef857073a55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4549046f27d64dc096cf0be2e0a3ed93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 16:36:46.992918 13284 tablet_service.cc:1467] Processing CreateTablet for tablet 03e5cea41ef34868b96f5ef857073a55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4549046f27d64dc096cf0be2e0a3ed93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 16:36:46.994045 13284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e5cea41ef34868b96f5ef857073a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:46.994035 13143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e5cea41ef34868b96f5ef857073a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:46.997803 13425 tablet_service.cc:1467] Processing CreateTablet for tablet 03e5cea41ef34868b96f5ef857073a55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4549046f27d64dc096cf0be2e0a3ed93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 16:36:46.999151 13425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e5cea41ef34868b96f5ef857073a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:47.000691 13515 tablet_bootstrap.cc:492] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: Bootstrap starting.
I20260928 16:36:47.000959 13516 tablet_bootstrap.cc:492] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: Bootstrap starting.
I20260928 16:36:47.002722 13516 tablet_bootstrap.cc:654] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:47.002722 13515 tablet_bootstrap.cc:654] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:47.006583 13516 tablet_bootstrap.cc:492] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: No bootstrap required, opened a new log
I20260928 16:36:47.006754 13514 tablet_bootstrap.cc:492] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: Bootstrap starting.
I20260928 16:36:47.006762 13516 ts_tablet_manager.cc:1397] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260928 16:36:47.007669 13516 raft_consensus.cc:348] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.007968 13516 raft_consensus.cc:374] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:47.008052 13516 raft_consensus.cc:729] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066d7245e5a946aeb7b0c20b62816dd4, State: Initialized, Role: FOLLOWER
I20260928 16:36:47.008325 13516 consensus_queue.cc:260] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.008764 13514 tablet_bootstrap.cc:654] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:47.009091 13516 ts_tablet_manager.cc:1428] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 16:36:47.008901 13521 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.009392 13521 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:47.009653 13515 tablet_bootstrap.cc:492] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: No bootstrap required, opened a new log
I20260928 16:36:47.009824 13515 ts_tablet_manager.cc:1397] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.003s
I20260928 16:36:47.011597 13514 tablet_bootstrap.cc:492] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: No bootstrap required, opened a new log
I20260928 16:36:47.011840 13514 ts_tablet_manager.cc:1397] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260928 16:36:47.012022 13515 raft_consensus.cc:348] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.012318 13515 raft_consensus.cc:374] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:47.012422 13515 raft_consensus.cc:729] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf41783244904579a96610aa89263e42, State: Initialized, Role: FOLLOWER
I20260928 16:36:47.012723 13515 consensus_queue.cc:260] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.012686 13514 raft_consensus.cc:348] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.012915 13514 raft_consensus.cc:374] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:47.013012 13514 raft_consensus.cc:729] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75bf71c13314e26b2c2e5f1f7738d8a, State: Initialized, Role: FOLLOWER
I20260928 16:36:47.013362 13515 ts_tablet_manager.cc:1428] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 16:36:47.013202 13522 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.013537 13522 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:47.013378 13514 consensus_queue.cc:260] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.013931 13520 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.014343 13520 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:47.014762 13514 ts_tablet_manager.cc:1428] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 16:36:47.048419 13522 raft_consensus.cc:482] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:36:47.048718 13522 raft_consensus.cc:504] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.049582 13522 leader_election.cc:288] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205), a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607)
I20260928 16:36:47.050330 13304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e5cea41ef34868b96f5ef857073a55" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4" is_pre_election: true
I20260928 16:36:47.050463 13445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e5cea41ef34868b96f5ef857073a55" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" is_pre_election: true
I20260928 16:36:47.050673 13304 raft_consensus.cc:2454] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 0.
I20260928 16:36:47.050800 13445 raft_consensus.cc:2454] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 0.
I20260928 16:36:47.051151 13095 leader_election.cc:302] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066d7245e5a946aeb7b0c20b62816dd4, bf41783244904579a96610aa89263e42; no voters:
I20260928 16:36:47.051492 13522 raft_consensus.cc:2781] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:36:47.051587 13522 raft_consensus.cc:482] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:36:47.051700 13522 raft_consensus.cc:3037] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:47.054196 13522 raft_consensus.cc:504] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.054804 13522 leader_election.cc:288] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 election: Requested vote from peers 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205), a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607)
I20260928 16:36:47.055408 13304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e5cea41ef34868b96f5ef857073a55" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4"
I20260928 16:36:47.055408 13445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e5cea41ef34868b96f5ef857073a55" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a"
I20260928 16:36:47.055662 13304 raft_consensus.cc:3037] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:47.055672 13445 raft_consensus.cc:3037] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:47.058260 13304 raft_consensus.cc:2454] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 1.
I20260928 16:36:47.058462 13445 raft_consensus.cc:2454] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 1.
I20260928 16:36:47.058765 13095 leader_election.cc:302] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066d7245e5a946aeb7b0c20b62816dd4, bf41783244904579a96610aa89263e42; no voters:
I20260928 16:36:47.059198 13522 raft_consensus.cc:2781] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:47.059402 13522 raft_consensus.cc:686] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [term 1 LEADER]: Becoming Leader. State: Replica: bf41783244904579a96610aa89263e42, State: Running, Role: LEADER
I20260928 16:36:47.059692 13522 consensus_queue.cc:237] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.061636 13525 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: TxnStatusTablet state changed. Reason: New leader bf41783244904579a96610aa89263e42. Latest consensus state: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.062001 13525 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: This TxnStatusTablet replica's current role is: LEADER
I20260928 16:36:47.062842 13537 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 16:36:47.063253 13537 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 16:36:47.063687 13004 catalog_manager.cc:5495] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf41783244904579a96610aa89263e42 (127.10.247.193). New cstate: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } health_report { overall_health: UNKNOWN } } }
I20260928 16:36:47.099516 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:47.092636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:34131 (local address 127.0.0.1:45620)
0928 16:36:47.092892 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:47.092915 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:47.093050 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:47.093624 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:47.093628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:47.093645 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 16:36:47.093852 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:47.093861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.095337 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.095342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:47.096442 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:47.096457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.097510 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.097515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:47.097714 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:47.097792 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 16:36:47.099025 (+ 1233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 16:36:47.099040 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 16:36:47.099092 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 16:36:47.099237 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 16:36:47.099592 13508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:47.092729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45620 (local address 127.10.247.193:34131)
0928 16:36:47.092977 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:47.092983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:47.093009 (+ 26us) server_negotiation.cc:407] Connection header received
0928 16:36:47.093287 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:47.093294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:47.093441 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:47.093611 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:47.093989 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.095197 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.096591 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.097356 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.097495 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:47.098023 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:47.098926 (+ 903us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:47.099115 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:47.099168 (+ 53us) server_negotiation.cc:299] Negotiation successful
0928 16:36:47.099270 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260928 16:36:47.110508 13445 raft_consensus.cc:1261] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Refusing update from remote peer bf41783244904579a96610aa89263e42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:47.110566 13304 raft_consensus.cc:1261] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Refusing update from remote peer bf41783244904579a96610aa89263e42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:47.111786 13525 consensus_queue.cc:1035] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:47.112267 13522 consensus_queue.cc:1035] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:47.125895 13520 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: TxnStatusTablet state changed. Reason: New leader bf41783244904579a96610aa89263e42. Latest consensus state: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.126307 13520 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:47.127840 13521 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: TxnStatusTablet state changed. Reason: New leader bf41783244904579a96610aa89263e42. Latest consensus state: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.128202 13521 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:47.131659 13540 mvcc.cc:205] Tried to move back new op lower bound from 7334352515512078336 to 7334352515319205888. Current Snapshot: MvccSnapshot[applied={T|T < 7334352515512078336}]
I20260928 16:36:47.131839 13542 mvcc.cc:205] Tried to move back new op lower bound from 7334352515512078336 to 7334352515319205888. Current Snapshot: MvccSnapshot[applied={T|T < 7334352515512078336}]
I20260928 16:36:47.134271 13520 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.134557 13520 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P a75bf71c13314e26b2c2e5f1f7738d8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:47.141707 13521 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.142024 13521 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P 066d7245e5a946aeb7b0c20b62816dd4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:36:47.142793 13525 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.143113 13525 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: This TxnStatusTablet replica's current role is: LEADER
I20260928 16:36:47.144059 13525 tablet_replica.cc:439] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } } }
I20260928 16:36:47.144330 13525 tablet_replica.cc:442] T 03e5cea41ef34868b96f5ef857073a55 P bf41783244904579a96610aa89263e42: This TxnStatusTablet replica's current role is: LEADER
I20260928 16:36:47.157600 13508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:47.151509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.10.247.193:34131)
0928 16:36:47.151688 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:47.151694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:47.152267 (+ 573us) server_negotiation.cc:407] Connection header received
0928 16:36:47.152481 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:47.152485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:47.152566 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:47.152696 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:47.153234 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.154132 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.155604 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.156118 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.156269 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:47.156737 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:47.156892 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:47.157001 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:47.157124 (+ 123us) server_negotiation.cc:299] Negotiation successful
0928 16:36:47.157226 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 16:36:47.180464 13508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:47.172896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:39055 (local address 127.10.247.193:34131)
0928 16:36:47.173143 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:47.173152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:47.173799 (+ 647us) server_negotiation.cc:407] Connection header received
0928 16:36:47.174041 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:47.174046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:47.174125 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:47.174214 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:47.174952 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.175833 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.178247 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.178793 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.178900 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:47.179633 (+ 733us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:47.179782 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:47.179849 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:47.180012 (+ 163us) server_negotiation.cc:299] Negotiation successful
0928 16:36:47.180155 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":64}
I20260928 16:36:47.180881 13553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:47.172787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:34131 (local address 127.10.247.193:39055)
0928 16:36:47.173653 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:47.173678 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:47.173900 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:47.174249 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:47.174254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:47.174268 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 16:36:47.174751 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:47.174766 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.175981 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.175987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:47.178070 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:47.178093 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:47.179001 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:47.179008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:47.179230 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:47.179338 (+ 108us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 16:36:47.179880 (+ 542us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 16:36:47.179893 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 16:36:47.179981 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 16:36:47.180266 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":39,"thread_start_us":441,"threads_started":1}
I20260928 16:36:47.194042 13445 raft_consensus.cc:1261] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Refusing update from remote peer bf41783244904579a96610aa89263e42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:47.194042 13303 raft_consensus.cc:1261] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Refusing update from remote peer bf41783244904579a96610aa89263e42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:47.194748 13525 consensus_queue.cc:1035] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:47.195117 13522 consensus_queue.cc:1035] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:47.280063 13004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:47.280402 13004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:47.284875 13004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd: Sending DeleteTablet for 3 replicas of tablet 96b7738b758f4cf58fb8bd4f8c8c765e
I20260928 16:36:47.286546 13284 tablet_service.cc:1514] Processing DeleteTablet for tablet 96b7738b758f4cf58fb8bd4f8c8c765e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:47 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:55442
I20260928 16:36:47.286552 13142 tablet_service.cc:1514] Processing DeleteTablet for tablet 96b7738b758f4cf58fb8bd4f8c8c765e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:47 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:45614
I20260928 16:36:47.286653 13425 tablet_service.cc:1514] Processing DeleteTablet for tablet 96b7738b758f4cf58fb8bd4f8c8c765e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:47 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:36582
I20260928 16:36:47.289695 13004 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:36:47.291851 13004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:36:47.296736 13561 tablet_replica.cc:330] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: stopping tablet replica
I20260928 16:36:47.296856 13560 tablet_replica.cc:330] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: stopping tablet replica
I20260928 16:36:47.297504 13560 raft_consensus.cc:2229] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:47.298003 13560 raft_consensus.cc:2258] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:47.300712 13559 tablet_replica.cc:330] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: stopping tablet replica
I20260928 16:36:47.301223 13559 raft_consensus.cc:2229] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:47.301508 13142 tablet_service.cc:1467] Processing CreateTablet for tablet 782898d5c3784042988ed7c4012ef674 (DEFAULT_TABLE table=test-table [id=26881b6334ff4d939a7def8b6a757a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:47.301668 13559 raft_consensus.cc:2258] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:47.301651 13560 ts_tablet_manager.cc:1905] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:47.302183 13142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 782898d5c3784042988ed7c4012ef674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:47.303777 13559 ts_tablet_manager.cc:1905] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:47.307710 13425 tablet_service.cc:1467] Processing CreateTablet for tablet 782898d5c3784042988ed7c4012ef674 (DEFAULT_TABLE table=test-table [id=26881b6334ff4d939a7def8b6a757a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:47.308265 13425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 782898d5c3784042988ed7c4012ef674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:47.311707 13559 ts_tablet_manager.cc:1918] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:36:47.311920 13559 log.cc:1196] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/96b7738b758f4cf58fb8bd4f8c8c765e
I20260928 16:36:47.312481 13559 ts_tablet_manager.cc:1939] T 96b7738b758f4cf58fb8bd4f8c8c765e P 066d7245e5a946aeb7b0c20b62816dd4: Deleting consensus metadata
I20260928 16:36:47.312911 13561 raft_consensus.cc:2229] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:36:47.313886 13561 raft_consensus.cc:2258] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:47.315937 13514 tablet_bootstrap.cc:492] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: Bootstrap starting.
I20260928 16:36:47.316540 12992 catalog_manager.cc:4841] TS 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205): tablet 96b7738b758f4cf58fb8bd4f8c8c765e (table test-table [id=8dff93ea5d434fc8b975b69103d47bcb]) successfully deleted
I20260928 16:36:47.317339 13561 ts_tablet_manager.cc:1905] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:47.318236 13514 tablet_bootstrap.cc:654] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:47.319977 13515 tablet_bootstrap.cc:492] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: Bootstrap starting.
I20260928 16:36:47.320762 13284 tablet_service.cc:1467] Processing CreateTablet for tablet 782898d5c3784042988ed7c4012ef674 (DEFAULT_TABLE table=test-table [id=26881b6334ff4d939a7def8b6a757a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:47.321293 13284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 782898d5c3784042988ed7c4012ef674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:47.321914 13515 tablet_bootstrap.cc:654] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:47.328639 13516 tablet_bootstrap.cc:492] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: Bootstrap starting.
I20260928 16:36:47.330077 13560 ts_tablet_manager.cc:1918] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:36:47.330449 13560 log.cc:1196] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/96b7738b758f4cf58fb8bd4f8c8c765e
I20260928 16:36:47.330592 13516 tablet_bootstrap.cc:654] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:47.330886 13515 tablet_bootstrap.cc:492] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: No bootstrap required, opened a new log
I20260928 16:36:47.331125 13515 ts_tablet_manager.cc:1397] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260928 16:36:47.331346 13560 ts_tablet_manager.cc:1939] T 96b7738b758f4cf58fb8bd4f8c8c765e P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting consensus metadata
I20260928 16:36:47.333796 13515 raft_consensus.cc:348] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.334158 13515 raft_consensus.cc:374] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:47.334251 13515 raft_consensus.cc:729] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf41783244904579a96610aa89263e42, State: Initialized, Role: FOLLOWER
I20260928 16:36:47.334600 13561 ts_tablet_manager.cc:1918] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:36:47.334631 13515 consensus_queue.cc:260] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.335093 13561 log.cc:1196] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/96b7738b758f4cf58fb8bd4f8c8c765e
I20260928 16:36:47.335392 13515 ts_tablet_manager.cc:1428] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260928 16:36:47.336220 13561 ts_tablet_manager.cc:1939] T 96b7738b758f4cf58fb8bd4f8c8c765e P bf41783244904579a96610aa89263e42: Deleting consensus metadata
I20260928 16:36:47.337050 12992 catalog_manager.cc:4841] TS a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607): tablet 96b7738b758f4cf58fb8bd4f8c8c765e (table test-table [id=8dff93ea5d434fc8b975b69103d47bcb]) successfully deleted
I20260928 16:36:47.337604 13514 tablet_bootstrap.cc:492] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: No bootstrap required, opened a new log
I20260928 16:36:47.337853 13514 ts_tablet_manager.cc:1397] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.005s
I20260928 16:36:47.337908 13516 tablet_bootstrap.cc:492] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: No bootstrap required, opened a new log
I20260928 16:36:47.338133 13516 ts_tablet_manager.cc:1397] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260928 16:36:47.338847 13514 raft_consensus.cc:348] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.339177 13514 raft_consensus.cc:374] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:47.339282 13514 raft_consensus.cc:729] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75bf71c13314e26b2c2e5f1f7738d8a, State: Initialized, Role: FOLLOWER
I20260928 16:36:47.338984 13516 raft_consensus.cc:348] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.339588 13514 consensus_queue.cc:260] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.339887 13516 raft_consensus.cc:374] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:47.339982 13516 raft_consensus.cc:729] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066d7245e5a946aeb7b0c20b62816dd4, State: Initialized, Role: FOLLOWER
I20260928 16:36:47.340322 13516 consensus_queue.cc:260] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.341042 12991 catalog_manager.cc:4841] TS bf41783244904579a96610aa89263e42 (127.10.247.193:34131): tablet 96b7738b758f4cf58fb8bd4f8c8c765e (table test-table [id=8dff93ea5d434fc8b975b69103d47bcb]) successfully deleted
I20260928 16:36:47.342114 13516 ts_tablet_manager.cc:1428] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260928 16:36:47.342500 13514 ts_tablet_manager.cc:1428] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260928 16:36:47.394069 13558 raft_consensus.cc:482] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:36:47.394367 13558 raft_consensus.cc:504] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.395203 13558 leader_election.cc:288] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205), a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607)
I20260928 16:36:47.395833 13304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "782898d5c3784042988ed7c4012ef674" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4" is_pre_election: true
I20260928 16:36:47.395973 13445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "782898d5c3784042988ed7c4012ef674" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" is_pre_election: true
I20260928 16:36:47.396202 13304 raft_consensus.cc:2454] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 0.
I20260928 16:36:47.396271 13445 raft_consensus.cc:2454] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 0.
I20260928 16:36:47.396766 13095 leader_election.cc:302] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066d7245e5a946aeb7b0c20b62816dd4, bf41783244904579a96610aa89263e42; no voters:
I20260928 16:36:47.397075 13558 raft_consensus.cc:2781] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:36:47.397212 13558 raft_consensus.cc:482] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:36:47.397317 13558 raft_consensus.cc:3037] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:47.399763 13558 raft_consensus.cc:504] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.400411 13558 leader_election.cc:288] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 election: Requested vote from peers 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205), a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607)
I20260928 16:36:47.401096 13304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "782898d5c3784042988ed7c4012ef674" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4"
I20260928 16:36:47.401115 13445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "782898d5c3784042988ed7c4012ef674" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a"
I20260928 16:36:47.401360 13445 raft_consensus.cc:3037] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:47.401386 13304 raft_consensus.cc:3037] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:47.404675 13304 raft_consensus.cc:2454] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 1.
I20260928 16:36:47.404690 13445 raft_consensus.cc:2454] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 1.
I20260928 16:36:47.405226 13095 leader_election.cc:302] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a75bf71c13314e26b2c2e5f1f7738d8a, bf41783244904579a96610aa89263e42; no voters:
I20260928 16:36:47.405567 13558 raft_consensus.cc:2781] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:47.405767 13558 raft_consensus.cc:686] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 1 LEADER]: Becoming Leader. State: Replica: bf41783244904579a96610aa89263e42, State: Running, Role: LEADER
I20260928 16:36:47.406164 13558 consensus_queue.cc:237] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:47.409642 13005 catalog_manager.cc:5495] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf41783244904579a96610aa89263e42 (127.10.247.193). New cstate: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } health_report { overall_health: UNKNOWN } } }
I20260928 16:36:47.463434 13303 raft_consensus.cc:1261] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Refusing update from remote peer bf41783244904579a96610aa89263e42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:47.463734 13444 raft_consensus.cc:1261] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Refusing update from remote peer bf41783244904579a96610aa89263e42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:47.464078 13525 consensus_queue.cc:1035] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:47.464381 13556 consensus_queue.cc:1035] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:47.471901 13543 mvcc.cc:205] Tried to move back new op lower bound from 7334352516961705984 to 7334352516738146304. Current Snapshot: MvccSnapshot[applied={T|T < 7334352516961705984}]
W20260928 16:36:47.492841 13540 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 16:36:49.092046 13005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:49.092384 13005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:49.097865 13005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd: Sending DeleteTablet for 3 replicas of tablet 782898d5c3784042988ed7c4012ef674
I20260928 16:36:49.098950 13284 tablet_service.cc:1514] Processing DeleteTablet for tablet 782898d5c3784042988ed7c4012ef674 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:49 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:55442
I20260928 16:36:49.098955 13143 tablet_service.cc:1514] Processing DeleteTablet for tablet 782898d5c3784042988ed7c4012ef674 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:49 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:45614
I20260928 16:36:49.099056 13425 tablet_service.cc:1514] Processing DeleteTablet for tablet 782898d5c3784042988ed7c4012ef674 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:49 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:36582
I20260928 16:36:49.112726 13588 tablet_replica.cc:330] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: stopping tablet replica
I20260928 16:36:49.113029 13588 raft_consensus.cc:2229] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:49.113358 13588 raft_consensus.cc:2258] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:49.114185 13589 tablet_replica.cc:330] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: stopping tablet replica
I20260928 16:36:49.114435 13589 raft_consensus.cc:2229] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:36:49.114769 13590 tablet_replica.cc:330] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: stopping tablet replica
I20260928 16:36:49.114924 13589 raft_consensus.cc:2258] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:49.115002 13590 raft_consensus.cc:2229] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:49.115447 13590 raft_consensus.cc:2258] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:49.116101 13588 ts_tablet_manager.cc:1905] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:49.116731 13590 ts_tablet_manager.cc:1905] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:49.121562 13589 ts_tablet_manager.cc:1905] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:49.122723 13590 ts_tablet_manager.cc:1918] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 16:36:49.122964 13590 log.cc:1196] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/782898d5c3784042988ed7c4012ef674
I20260928 16:36:49.123569 13590 ts_tablet_manager.cc:1939] T 782898d5c3784042988ed7c4012ef674 P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting consensus metadata
I20260928 16:36:49.123678 13588 ts_tablet_manager.cc:1918] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 16:36:49.123932 13588 log.cc:1196] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/782898d5c3784042988ed7c4012ef674
I20260928 16:36:49.124555 13588 ts_tablet_manager.cc:1939] T 782898d5c3784042988ed7c4012ef674 P 066d7245e5a946aeb7b0c20b62816dd4: Deleting consensus metadata
I20260928 16:36:49.125140 12992 catalog_manager.cc:4841] TS a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607): tablet 782898d5c3784042988ed7c4012ef674 (table test-table [id=26881b6334ff4d939a7def8b6a757a87]) successfully deleted
I20260928 16:36:49.126003 12992 catalog_manager.cc:4841] TS 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205): tablet 782898d5c3784042988ed7c4012ef674 (table test-table [id=26881b6334ff4d939a7def8b6a757a87]) successfully deleted
I20260928 16:36:49.128664 13589 ts_tablet_manager.cc:1918] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 16:36:49.128803 13589 log.cc:1196] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/782898d5c3784042988ed7c4012ef674
I20260928 16:36:49.129194 13589 ts_tablet_manager.cc:1939] T 782898d5c3784042988ed7c4012ef674 P bf41783244904579a96610aa89263e42: Deleting consensus metadata
I20260928 16:36:49.130329 12991 catalog_manager.cc:4841] TS bf41783244904579a96610aa89263e42 (127.10.247.193:34131): tablet 782898d5c3784042988ed7c4012ef674 (table test-table [id=26881b6334ff4d939a7def8b6a757a87]) successfully deleted
Sep 28 16:36:49 dist-test-slave-9fn6 krb5kdc[12959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 16:36:49 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:49.167060 13598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.149136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.10.247.254:46103)
0928 16:36:49.150153 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:49.150159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:49.150182 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:36:49.150253 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:49.150257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:49.150329 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:49.150433 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:49.152136 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.153165 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.154677 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.155187 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.155321 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.160451 (+ 5130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:49.160458 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:49.160462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:49.160552 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:49.163779 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:49.164702 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:49.164709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:49.164711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:49.164876 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:49.165367 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:49.165372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:49.165374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:49.165761 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:49.165987 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:49.166397 (+ 410us) server_negotiation.cc:299] Negotiation successful
0928 16:36:49.166645 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":714,"threads_started":1}
I20260928 16:36:49.171537 13005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:36:49.172454 13005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:36:49.181250 13143 tablet_service.cc:1467] Processing CreateTablet for tablet 89a2293ec9f344d3a1378227c84a1e13 (DEFAULT_TABLE table=test-table [id=fd1ae91246854113832b05b135958cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:49.181286 13284 tablet_service.cc:1467] Processing CreateTablet for tablet 89a2293ec9f344d3a1378227c84a1e13 (DEFAULT_TABLE table=test-table [id=fd1ae91246854113832b05b135958cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:49.181514 13425 tablet_service.cc:1467] Processing CreateTablet for tablet 89a2293ec9f344d3a1378227c84a1e13 (DEFAULT_TABLE table=test-table [id=fd1ae91246854113832b05b135958cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:49.181833 13284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a2293ec9f344d3a1378227c84a1e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:49.181849 13143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a2293ec9f344d3a1378227c84a1e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:49.181991 13425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a2293ec9f344d3a1378227c84a1e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:49.189338 13599 tablet_bootstrap.cc:492] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: Bootstrap starting.
I20260928 16:36:49.190050 13600 tablet_bootstrap.cc:492] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: Bootstrap starting.
I20260928 16:36:49.190773 13601 tablet_bootstrap.cc:492] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: Bootstrap starting.
I20260928 16:36:49.192189 13599 tablet_bootstrap.cc:654] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:49.192967 13601 tablet_bootstrap.cc:654] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:49.193002 13600 tablet_bootstrap.cc:654] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:49.195093 13599 tablet_bootstrap.cc:492] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: No bootstrap required, opened a new log
I20260928 16:36:49.195272 13601 tablet_bootstrap.cc:492] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: No bootstrap required, opened a new log
I20260928 16:36:49.195277 13599 ts_tablet_manager.cc:1397] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 16:36:49.195287 13600 tablet_bootstrap.cc:492] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: No bootstrap required, opened a new log
I20260928 16:36:49.195422 13601 ts_tablet_manager.cc:1397] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260928 16:36:49.195446 13600 ts_tablet_manager.cc:1397] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 16:36:49.196282 13599 raft_consensus.cc:348] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.196599 13599 raft_consensus.cc:374] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:49.196507 13601 raft_consensus.cc:348] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.196754 13599 raft_consensus.cc:729] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf41783244904579a96610aa89263e42, State: Initialized, Role: FOLLOWER
I20260928 16:36:49.196805 13601 raft_consensus.cc:374] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:49.196593 13600 raft_consensus.cc:348] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.196892 13601 raft_consensus.cc:729] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75bf71c13314e26b2c2e5f1f7738d8a, State: Initialized, Role: FOLLOWER
I20260928 16:36:49.196908 13600 raft_consensus.cc:374] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:49.197006 13600 raft_consensus.cc:729] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066d7245e5a946aeb7b0c20b62816dd4, State: Initialized, Role: FOLLOWER
I20260928 16:36:49.197027 13599 consensus_queue.cc:260] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.197153 13601 consensus_queue.cc:260] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.197687 13599 ts_tablet_manager.cc:1428] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 16:36:49.198010 13601 ts_tablet_manager.cc:1428] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 16:36:49.197254 13600 consensus_queue.cc:260] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.199407 13600 ts_tablet_manager.cc:1428] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260928 16:36:49.421540 13576 raft_consensus.cc:482] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:36:49.421820 13576 raft_consensus.cc:504] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.422730 13576 leader_election.cc:288] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205), a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607)
I20260928 16:36:49.423589 13304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a2293ec9f344d3a1378227c84a1e13" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4" is_pre_election: true
I20260928 16:36:49.423631 13444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a2293ec9f344d3a1378227c84a1e13" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" is_pre_election: true
I20260928 16:36:49.423955 13444 raft_consensus.cc:2454] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 0.
I20260928 16:36:49.423961 13304 raft_consensus.cc:2454] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 0.
I20260928 16:36:49.424633 13095 leader_election.cc:302] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066d7245e5a946aeb7b0c20b62816dd4, bf41783244904579a96610aa89263e42; no voters:
I20260928 16:36:49.424955 13576 raft_consensus.cc:2781] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:36:49.425123 13576 raft_consensus.cc:482] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:36:49.425230 13576 raft_consensus.cc:3037] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:49.428156 13576 raft_consensus.cc:504] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.428862 13576 leader_election.cc:288] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 election: Requested vote from peers 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205), a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607)
I20260928 16:36:49.429472 13304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a2293ec9f344d3a1378227c84a1e13" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4"
I20260928 16:36:49.429574 13444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a2293ec9f344d3a1378227c84a1e13" candidate_uuid: "bf41783244904579a96610aa89263e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a"
I20260928 16:36:49.429817 13304 raft_consensus.cc:3037] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:49.429852 13444 raft_consensus.cc:3037] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:49.432283 13444 raft_consensus.cc:2454] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 1.
I20260928 16:36:49.432554 13304 raft_consensus.cc:2454] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf41783244904579a96610aa89263e42 in term 1.
I20260928 16:36:49.432845 13095 leader_election.cc:302] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a75bf71c13314e26b2c2e5f1f7738d8a, bf41783244904579a96610aa89263e42; no voters:
I20260928 16:36:49.433229 13576 raft_consensus.cc:2781] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:49.433399 13576 raft_consensus.cc:686] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 1 LEADER]: Becoming Leader. State: Replica: bf41783244904579a96610aa89263e42, State: Running, Role: LEADER
I20260928 16:36:49.433713 13576 consensus_queue.cc:237] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.436887 13005 catalog_manager.cc:5495] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf41783244904579a96610aa89263e42 (127.10.247.193). New cstate: current_term: 1 leader_uuid: "bf41783244904579a96610aa89263e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } health_report { overall_health: UNKNOWN } } }
I20260928 16:36:49.503723 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.498101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45628 (local address 127.10.247.193:34131)
0928 16:36:49.499084 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:49.499088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:49.499109 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:36:49.499166 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:49.499169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:49.499240 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:49.499326 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:49.499682 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.500556 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.501935 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.502384 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.502500 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.502914 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:49.503042 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:49.503106 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:49.503223 (+ 117us) server_negotiation.cc:299] Negotiation successful
0928 16:36:49.503293 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":710,"threads_started":1}
I20260928 16:36:49.513945 13304 raft_consensus.cc:1261] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Refusing update from remote peer bf41783244904579a96610aa89263e42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:49.514012 13444 raft_consensus.cc:1261] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Refusing update from remote peer bf41783244904579a96610aa89263e42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:49.514526 13525 consensus_queue.cc:1035] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:49.514846 13570 consensus_queue.cc:1035] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:49.566901 13005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:49.567230 13005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:49.570855 13005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd: Sending DeleteTablet for 3 replicas of tablet 89a2293ec9f344d3a1378227c84a1e13
I20260928 16:36:49.571833 13143 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a2293ec9f344d3a1378227c84a1e13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:49 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:45614
I20260928 16:36:49.571837 13284 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a2293ec9f344d3a1378227c84a1e13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:49 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:55442
I20260928 16:36:49.571914 13425 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a2293ec9f344d3a1378227c84a1e13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:49 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:36582
I20260928 16:36:49.572192 13588 tablet_replica.cc:330] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: stopping tablet replica
I20260928 16:36:49.572293 13589 tablet_replica.cc:330] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: stopping tablet replica
I20260928 16:36:49.572497 13590 tablet_replica.cc:330] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: stopping tablet replica
I20260928 16:36:49.572465 13588 raft_consensus.cc:2229] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:49.572567 13589 raft_consensus.cc:2229] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:36:49.572755 13590 raft_consensus.cc:2229] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:49.572788 13588 raft_consensus.cc:2258] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:49.572971 13589 raft_consensus.cc:2258] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:49.573016 13590 raft_consensus.cc:2258] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:49.573974 13005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 16:36:49.574311 13589 ts_tablet_manager.cc:1905] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:49.574653 13588 ts_tablet_manager.cc:1905] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 16:36:49.574790 13005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:36:49.575217 13590 ts_tablet_manager.cc:1905] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:49.584197 13143 tablet_service.cc:1467] Processing CreateTablet for tablet d378242dbb4b4bddbd1ea0e579790629 (DEFAULT_TABLE table=test-table [id=cbdfd6bef08547d493ff28cdacc3f181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:49.584173 13284 tablet_service.cc:1467] Processing CreateTablet for tablet d378242dbb4b4bddbd1ea0e579790629 (DEFAULT_TABLE table=test-table [id=cbdfd6bef08547d493ff28cdacc3f181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:49.585238 13425 tablet_service.cc:1467] Processing CreateTablet for tablet d378242dbb4b4bddbd1ea0e579790629 (DEFAULT_TABLE table=test-table [id=cbdfd6bef08547d493ff28cdacc3f181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:36:49.609076 13425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d378242dbb4b4bddbd1ea0e579790629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:49.609154 13590 ts_tablet_manager.cc:1918] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:36:49.609169 13589 ts_tablet_manager.cc:1918] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:36:49.609313 13590 log.cc:1196] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/89a2293ec9f344d3a1378227c84a1e13
I20260928 16:36:49.609370 13589 log.cc:1196] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/89a2293ec9f344d3a1378227c84a1e13
I20260928 16:36:49.609840 13589 ts_tablet_manager.cc:1939] T 89a2293ec9f344d3a1378227c84a1e13 P bf41783244904579a96610aa89263e42: Deleting consensus metadata
I20260928 16:36:49.610059 13590 ts_tablet_manager.cc:1939] T 89a2293ec9f344d3a1378227c84a1e13 P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting consensus metadata
I20260928 16:36:49.611322 13143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d378242dbb4b4bddbd1ea0e579790629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:49.611624 12992 catalog_manager.cc:4841] TS a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607): tablet 89a2293ec9f344d3a1378227c84a1e13 (table test-table [id=fd1ae91246854113832b05b135958cbf]) successfully deleted
I20260928 16:36:49.614607 13284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d378242dbb4b4bddbd1ea0e579790629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:49.617928 13601 tablet_bootstrap.cc:492] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: Bootstrap starting.
I20260928 16:36:49.618407 13599 tablet_bootstrap.cc:492] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: Bootstrap starting.
I20260928 16:36:49.619915 13601 tablet_bootstrap.cc:654] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:49.620230 13599 tablet_bootstrap.cc:654] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:49.623296 12991 catalog_manager.cc:4841] TS bf41783244904579a96610aa89263e42 (127.10.247.193:34131): tablet 89a2293ec9f344d3a1378227c84a1e13 (table test-table [id=fd1ae91246854113832b05b135958cbf]) successfully deleted
I20260928 16:36:49.625368 13600 tablet_bootstrap.cc:492] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: Bootstrap starting.
I20260928 16:36:49.627556 13600 tablet_bootstrap.cc:654] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:49.632169 13601 tablet_bootstrap.cc:492] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: No bootstrap required, opened a new log
I20260928 16:36:49.632546 13601 ts_tablet_manager.cc:1397] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260928 16:36:49.632639 13599 tablet_bootstrap.cc:492] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: No bootstrap required, opened a new log
I20260928 16:36:49.632835 13600 tablet_bootstrap.cc:492] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: No bootstrap required, opened a new log
I20260928 16:36:49.632995 13599 ts_tablet_manager.cc:1397] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260928 16:36:49.633055 13600 ts_tablet_manager.cc:1397] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260928 16:36:49.633553 13601 raft_consensus.cc:348] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.633849 13601 raft_consensus.cc:374] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:49.633919 13601 raft_consensus.cc:729] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75bf71c13314e26b2c2e5f1f7738d8a, State: Initialized, Role: FOLLOWER
I20260928 16:36:49.633949 13599 raft_consensus.cc:348] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.634243 13599 raft_consensus.cc:374] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:49.634130 13600 raft_consensus.cc:348] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.634320 13599 raft_consensus.cc:729] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf41783244904579a96610aa89263e42, State: Initialized, Role: FOLLOWER
I20260928 16:36:49.634394 13600 raft_consensus.cc:374] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:49.634287 13601 consensus_queue.cc:260] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.634474 13600 raft_consensus.cc:729] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066d7245e5a946aeb7b0c20b62816dd4, State: Initialized, Role: FOLLOWER
I20260928 16:36:49.634596 13599 consensus_queue.cc:260] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.634734 13600 consensus_queue.cc:260] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.635210 13599 ts_tablet_manager.cc:1428] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260928 16:36:49.635314 13600 ts_tablet_manager.cc:1428] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260928 16:36:49.636459 13601 ts_tablet_manager.cc:1428] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260928 16:36:49.648586 13605 raft_consensus.cc:482] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:36:49.648860 13605 raft_consensus.cc:504] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.650723 13605 leader_election.cc:288] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf41783244904579a96610aa89263e42 (127.10.247.193:34131), 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205)
I20260928 16:36:49.662322 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.651211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:60353 (local address 127.10.247.194:38205)
0928 16:36:49.654146 (+ 2935us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:49.654152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:49.654589 (+ 437us) server_negotiation.cc:407] Connection header received
0928 16:36:49.654829 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:49.654835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:49.654929 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:49.655052 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:49.655749 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.656858 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.659984 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.661121 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.661405 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.661505 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:49.661701 (+ 196us) server_negotiation.cc:299] Negotiation successful
0928 16:36:49.661891 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2766,"thread_start_us":2600,"threads_started":1}
I20260928 16:36:49.662498 13617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.650994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:38205 (local address 127.10.247.195:60353)
0928 16:36:49.654453 (+ 3459us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:49.654478 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:49.654666 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:49.655087 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:49.655092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:49.655126 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:49.655538 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:49.655551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.657034 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.657038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:49.659795 (+ 2757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:49.659813 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.661334 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.661343 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:49.661518 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.661556 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 16:36:49.661700 (+ 144us) client_negotiation.cc:240] Negotiation successful
0928 16:36:49.661858 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3299,"mutex_wait_us":38,"thread_start_us":3065,"threads_started":1}
I20260928 16:36:49.662569 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.651196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:37597 (local address 127.10.247.193:34131)
0928 16:36:49.651405 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:49.651412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:49.654767 (+ 3355us) server_negotiation.cc:407] Connection header received
0928 16:36:49.654888 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:49.654915 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:49.655041 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:49.655131 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:36:49.656713 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.657684 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.660285 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.661422 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.661577 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.661658 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:49.662092 (+ 434us) server_negotiation.cc:299] Negotiation successful
0928 16:36:49.662231 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 16:36:49.662942 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.651151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:34131 (local address 127.10.247.195:37597)
0928 16:36:49.654653 (+ 3502us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:49.654669 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:49.654782 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:49.655327 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:49.655330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:49.655353 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:36:49.656057 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:49.656072 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.657823 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.657828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:49.660152 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:49.660164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.661721 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.661730 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:49.661921 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.661946 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 16:36:49.662038 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 16:36:49.662167 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3375,"thread_start_us":3145,"threads_started":1}
I20260928 16:36:49.663669 13303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d378242dbb4b4bddbd1ea0e579790629" candidate_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4" is_pre_election: true
I20260928 16:36:49.664052 13303 raft_consensus.cc:2454] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a75bf71c13314e26b2c2e5f1f7738d8a in term 0.
I20260928 16:36:49.664485 13163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d378242dbb4b4bddbd1ea0e579790629" candidate_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf41783244904579a96610aa89263e42" is_pre_election: true
I20260928 16:36:49.664593 13377 leader_election.cc:302] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066d7245e5a946aeb7b0c20b62816dd4, a75bf71c13314e26b2c2e5f1f7738d8a; no voters:
I20260928 16:36:49.665104 13605 raft_consensus.cc:2781] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:36:49.665256 13605 raft_consensus.cc:482] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:36:49.664989 13163 raft_consensus.cc:2454] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a75bf71c13314e26b2c2e5f1f7738d8a in term 0.
I20260928 16:36:49.665372 13605 raft_consensus.cc:3037] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:49.694248 13605 raft_consensus.cc:504] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.694394 13588 ts_tablet_manager.cc:1918] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:36:49.694646 13588 log.cc:1196] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/89a2293ec9f344d3a1378227c84a1e13
I20260928 16:36:49.695140 13588 ts_tablet_manager.cc:1939] T 89a2293ec9f344d3a1378227c84a1e13 P 066d7245e5a946aeb7b0c20b62816dd4: Deleting consensus metadata
I20260928 16:36:49.695760 13605 leader_election.cc:288] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [CANDIDATE]: Term 1 election: Requested vote from peers bf41783244904579a96610aa89263e42 (127.10.247.193:34131), 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205)
I20260928 16:36:49.695917 13303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d378242dbb4b4bddbd1ea0e579790629" candidate_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066d7245e5a946aeb7b0c20b62816dd4"
I20260928 16:36:49.696287 13303 raft_consensus.cc:3037] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:49.696419 13163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d378242dbb4b4bddbd1ea0e579790629" candidate_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf41783244904579a96610aa89263e42"
I20260928 16:36:49.696542 12992 catalog_manager.cc:4841] TS 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205): tablet 89a2293ec9f344d3a1378227c84a1e13 (table test-table [id=fd1ae91246854113832b05b135958cbf]) successfully deleted
I20260928 16:36:49.696728 13163 raft_consensus.cc:3037] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:49.791970 13303 raft_consensus.cc:2454] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a75bf71c13314e26b2c2e5f1f7738d8a in term 1.
I20260928 16:36:49.792008 13163 raft_consensus.cc:2454] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a75bf71c13314e26b2c2e5f1f7738d8a in term 1.
I20260928 16:36:49.792841 13376 leader_election.cc:302] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 066d7245e5a946aeb7b0c20b62816dd4, a75bf71c13314e26b2c2e5f1f7738d8a, bf41783244904579a96610aa89263e42; no voters:
I20260928 16:36:49.793305 13605 raft_consensus.cc:2781] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:49.794049 13605 raft_consensus.cc:686] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 LEADER]: Becoming Leader. State: Replica: a75bf71c13314e26b2c2e5f1f7738d8a, State: Running, Role: LEADER
I20260928 16:36:49.794572 13605 consensus_queue.cc:237] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } }
I20260928 16:36:49.799269 13004 catalog_manager.cc:5495] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a reported cstate change: term changed from 0 to 1, leader changed from <none> to a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195). New cstate: current_term: 1 leader_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75bf71c13314e26b2c2e5f1f7738d8a" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39607 } health_report { overall_health: HEALTHY } } }
I20260928 16:36:49.852391 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.845134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.10.247.195:39607)
0928 16:36:49.845801 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:49.845806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:49.845827 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:36:49.845941 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:49.845947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:49.846057 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:49.846222 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:49.846831 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.847762 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.849642 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.850312 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.850415 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.850874 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:49.851692 (+ 818us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:49.851815 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:49.851953 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 16:36:49.852045 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":352,"threads_started":1}
I20260928 16:36:49.872062 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.865674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:37819 (local address 127.10.247.193:34131)
0928 16:36:49.865909 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:49.865917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:49.866719 (+ 802us) server_negotiation.cc:407] Connection header received
0928 16:36:49.866934 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:49.866939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:49.867095 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:49.867178 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:49.867694 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.868528 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.870205 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.870655 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.870758 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.871312 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:49.871456 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:49.871536 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:49.871681 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 16:36:49.871763 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 16:36:49.872713 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.865667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:34131 (local address 127.10.247.195:37819)
0928 16:36:49.866574 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:49.866619 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:49.866795 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:49.867214 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:49.867218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:49.867229 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 16:36:49.867565 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:49.867574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.868701 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.868705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:49.870052 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:49.870066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.870817 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.870822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:49.871059 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.871144 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 16:36:49.871534 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 16:36:49.871580 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 16:36:49.871647 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 16:36:49.872198 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":373,"threads_started":1}
I20260928 16:36:49.883358 13303 raft_consensus.cc:1261] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Refusing update from remote peer a75bf71c13314e26b2c2e5f1f7738d8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:49.883636 13163 raft_consensus.cc:1261] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Refusing update from remote peer a75bf71c13314e26b2c2e5f1f7738d8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:36:49.884331 13620 consensus_queue.cc:1035] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "066d7245e5a946aeb7b0c20b62816dd4" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260928 16:36:49.884972 13605 consensus_queue.cc:1035] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf41783244904579a96610aa89263e42" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 34131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:36:49.923789 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.916079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:48947 (local address 127.10.247.195:39607)
0928 16:36:49.916291 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:49.916302 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:49.917070 (+ 768us) server_negotiation.cc:407] Connection header received
0928 16:36:49.917305 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:49.917311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:49.917396 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:49.917472 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:36:49.918681 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.919752 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.921478 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.922277 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.922405 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.922942 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:36:49.923135 (+ 193us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:36:49.923211 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:49.923384 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 16:36:49.923479 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 16:36:49.924084 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:49.915970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:39607 (local address 127.10.247.193:48947)
0928 16:36:49.916952 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:49.916970 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:49.917162 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:49.917501 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:49.917506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:49.917518 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 16:36:49.918466 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:49.918484 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.919884 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.919890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:49.921334 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:49.921348 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:49.922471 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:49.922479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:49.922634 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:49.922692 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 16:36:49.923239 (+ 547us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 16:36:49.923254 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 16:36:49.923333 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 16:36:49.923464 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":572,"threads_started":1}
W20260928 16:36:49.925613 13572 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 16:36:51.538094 13003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:51.538389 13003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:36:51.543834 13003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34f8cd927c03423d8d570136eec1f4cd: Sending DeleteTablet for 3 replicas of tablet d378242dbb4b4bddbd1ea0e579790629
I20260928 16:36:51.544909 13143 tablet_service.cc:1514] Processing DeleteTablet for tablet d378242dbb4b4bddbd1ea0e579790629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:51 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:45614
I20260928 16:36:51.544972 13284 tablet_service.cc:1514] Processing DeleteTablet for tablet d378242dbb4b4bddbd1ea0e579790629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:51 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:55442
I20260928 16:36:51.545042 13425 tablet_service.cc:1514] Processing DeleteTablet for tablet d378242dbb4b4bddbd1ea0e579790629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:36:51 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:36582
I20260928 16:36:51.545825 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13076
I20260928 16:36:51.549471 13656 tablet_replica.cc:330] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: stopping tablet replica
I20260928 16:36:51.550505 13655 tablet_replica.cc:330] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: stopping tablet replica
I20260928 16:36:51.549814 13656 raft_consensus.cc:2229] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:51.550861 13657 tablet_replica.cc:330] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: stopping tablet replica
I20260928 16:36:51.550843 13655 raft_consensus.cc:2229] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:36:51.551075 13656 raft_consensus.cc:2258] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:51.551122 13655 raft_consensus.cc:2258] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:51.551193 13657 raft_consensus.cc:2229] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:36:51.552075 13657 raft_consensus.cc:2258] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:36:51.552317 13656 ts_tablet_manager.cc:1905] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:51.553363 13655 ts_tablet_manager.cc:1905] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:51.553574 13657 ts_tablet_manager.cc:1905] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:36:51.560506 13656 ts_tablet_manager.cc:1918] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 16:36:51.560725 13656 log.cc:1196] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/d378242dbb4b4bddbd1ea0e579790629
I20260928 16:36:51.560920 13657 ts_tablet_manager.cc:1918] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 16:36:51.561036 13655 ts_tablet_manager.cc:1918] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 16:36:51.561098 13657 log.cc:1196] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/d378242dbb4b4bddbd1ea0e579790629
I20260928 16:36:51.561254 13655 log.cc:1196] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/d378242dbb4b4bddbd1ea0e579790629
I20260928 16:36:51.561259 13656 ts_tablet_manager.cc:1939] T d378242dbb4b4bddbd1ea0e579790629 P 066d7245e5a946aeb7b0c20b62816dd4: Deleting consensus metadata
I20260928 16:36:51.561610 13657 ts_tablet_manager.cc:1939] T d378242dbb4b4bddbd1ea0e579790629 P a75bf71c13314e26b2c2e5f1f7738d8a: Deleting consensus metadata
I20260928 16:36:51.561774 13655 ts_tablet_manager.cc:1939] T d378242dbb4b4bddbd1ea0e579790629 P bf41783244904579a96610aa89263e42: Deleting consensus metadata
I20260928 16:36:51.562808 12992 catalog_manager.cc:4841] TS 066d7245e5a946aeb7b0c20b62816dd4 (127.10.247.194:38205): tablet d378242dbb4b4bddbd1ea0e579790629 (table test-table [id=cbdfd6bef08547d493ff28cdacc3f181]) successfully deleted
I20260928 16:36:51.563184 12992 catalog_manager.cc:4841] TS a75bf71c13314e26b2c2e5f1f7738d8a (127.10.247.195:39607): tablet d378242dbb4b4bddbd1ea0e579790629 (table test-table [id=cbdfd6bef08547d493ff28cdacc3f181]) successfully deleted
I20260928 16:36:51.563191 12991 catalog_manager.cc:4841] TS bf41783244904579a96610aa89263e42 (127.10.247.193:34131): tablet d378242dbb4b4bddbd1ea0e579790629 (table test-table [id=cbdfd6bef08547d493ff28cdacc3f181]) successfully deleted
Sep 28 16:36:51 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:36:51.573068 13659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:51.547521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45638 (local address 127.10.247.193:34131)
0928 16:36:51.549617 (+ 2096us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:51.549623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:51.549650 (+ 27us) server_negotiation.cc:407] Connection header received
0928 16:36:51.549718 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:51.549723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:51.549814 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:51.549937 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:51.551794 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:51.552867 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:51.554753 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:51.555625 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:51.555740 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:51.566655 (+ 10915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:51.566675 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:51.566697 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:51.566746 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:51.570199 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:51.570884 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:51.570892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:51.570894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:51.571046 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:51.571424 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:51.571428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:51.571431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:51.571887 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:51.572216 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:51.572456 (+ 240us) server_negotiation.cc:299] Negotiation successful
0928 16:36:51.572726 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1945,"thread_start_us":1779,"threads_started":1}
W20260928 16:36:51.573850 13208 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45638
W20260928 16:36:51.574637 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13076: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:36:51.574945 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 13076
I20260928 16:36:51.610745 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13217
Sep 28 16:36:51 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:36:51.629103 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:51.611836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.10.247.194:38205)
0928 16:36:51.612572 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:51.612577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:51.612597 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:36:51.612699 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:51.612703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:51.612786 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:51.612902 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:51.614394 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:51.615225 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:51.616752 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:51.617251 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:51.617379 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:51.622648 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:51.622672 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:51.622694 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:51.622747 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:51.625956 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:51.626825 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:51.626836 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:51.626838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:51.627013 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:51.627396 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:51.627399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:51.627402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:51.627871 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:51.628144 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:51.628492 (+ 348us) server_negotiation.cc:299] Negotiation successful
0928 16:36:51.628777 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":351,"threads_started":1}
W20260928 16:36:51.629980 13349 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55444
W20260928 16:36:51.630739 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13217: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:36:51.630921 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 13217
I20260928 16:36:51.649338 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13358
Sep 28 16:36:51 dist-test-slave-9fn6 krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:36:51.666358 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:51.650331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.10.247.195:39607)
0928 16:36:51.650950 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:51.650956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:51.650974 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:36:51.651044 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:51.651048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:51.651122 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:51.651233 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:51.652858 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:51.653718 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:51.654988 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:51.655403 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:51.655496 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:51.660318 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:51.660340 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:51.660368 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:51.660425 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:51.663541 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:51.664247 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:51.664255 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:51.664258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:51.664430 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:51.664771 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:51.664774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:51.664777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:51.665184 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:51.665455 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:51.665771 (+ 316us) server_negotiation.cc:299] Negotiation successful
0928 16:36:51.665965 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":318,"threads_started":1}
W20260928 16:36:51.667198 13490 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53548
W20260928 16:36:51.667944 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13358: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:36:51.668131 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 13358
I20260928 16:36:51.690378 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 12973
I20260928 16:36:51.691900 13045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:51.824699 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 12973
2026-09-28T16:36:51Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8962 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:36:51 dist-test-slave-9fn6 krb5kdc[13665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:36:51 dist-test-slave-9fn6 krb5kdc[13665](info): set up 2 sockets
Sep 28 16:36:51 dist-test-slave-9fn6 krb5kdc[13665](info): commencing operation
krb5kdc: starting...
W20260928 16:36:52.849116 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:36:52 dist-test-slave-9fn6 krb5kdc[13665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:36:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:36:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:36:53.080004 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42053
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:42053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:53.457479 13678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:53.457858 13678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:53.458014 13678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:53.465956 13678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:36:53.466109 13678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:53.466172 13678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:53.466233 13678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:36:53.466279 13678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:36:53.477527 13678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:42053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:53.479286 13678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:53.480913 13678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:53.494742 13683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:53.494731 13684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:53.495007 13678 server_base.cc:1034] running on GCE node
W20260928 16:36:53.494717 13686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:53.496417 13678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:53.498214 13678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:53.499446 13678 hybrid_clock.cc:648] HybridClock initialized: now 1790613413499376 us; error 52 us; skew 500 ppm
I20260928 16:36:53.499818 13678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:53 dist-test-slave-9fn6 krb5kdc[13665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613413, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:53.503507 13678 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:36:53.505330 13678 webserver.cc:460] Webserver started at http://127.10.247.254:32851/ using document root <none> and password file <none>
I20260928 16:36:53.506198 13678 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:53.506322 13678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:53.506667 13678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:53.510295 13678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "bc6597ff40bb4b76858c8616adbefbb0"
format_stamp: "Formatted at 2026-09-28 16:36:53 on dist-test-slave-9fn6"
server_key: "eca4186c7c2be5a810ab2bc43c81ea3a"
server_key_iv: "496bf5aa91d10e74eeac552d6bbc7d9f"
server_key_version: "encryptionkey@0"
I20260928 16:36:53.511291 13678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "bc6597ff40bb4b76858c8616adbefbb0"
format_stamp: "Formatted at 2026-09-28 16:36:53 on dist-test-slave-9fn6"
server_key: "eca4186c7c2be5a810ab2bc43c81ea3a"
server_key_iv: "496bf5aa91d10e74eeac552d6bbc7d9f"
server_key_version: "encryptionkey@0"
I20260928 16:36:53.517804 13678 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260928 16:36:53.522258 13693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:53.524047 13678 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260928 16:36:53.524269 13678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "bc6597ff40bb4b76858c8616adbefbb0"
format_stamp: "Formatted at 2026-09-28 16:36:53 on dist-test-slave-9fn6"
server_key: "eca4186c7c2be5a810ab2bc43c81ea3a"
server_key_iv: "496bf5aa91d10e74eeac552d6bbc7d9f"
server_key_version: "encryptionkey@0"
I20260928 16:36:53.524519 13678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:53.555014 13678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:53.560155 13678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:53.560477 13678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:53.580082 13678 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:42053
I20260928 16:36:53.580090 13745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:42053 every 8 connection(s)
I20260928 16:36:53.581815 13678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:36:53.583680 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 13678
I20260928 16:36:53.583941 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:36:53.584904 11231 external_mini_cluster.cc:1428] Setting key c68e32465601cf823a8101ee16abc010
I20260928 16:36:53.587567 13746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 16:36:53 dist-test-slave-9fn6 krb5kdc[13665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:53.603052 13746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0: Bootstrap starting.
I20260928 16:36:53.608045 13746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:53.609438 13746 log.cc:824] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:53.612502 13749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:53.587343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.10.247.254:42053)
0928 16:36:53.588261 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:53.588278 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:53.588344 (+ 66us) server_negotiation.cc:407] Connection header received
0928 16:36:53.590901 (+ 2557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:53.590935 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:53.591560 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:53.592103 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:53.593732 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:53.595915 (+ 2183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:53.597450 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:53.598650 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:53.598872 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:53.603374 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:53.603386 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:53.603411 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:53.603470 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:53.607593 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:53.608494 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:53.608506 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:53.608521 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:53.608842 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:53.609302 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:53.609306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:53.609308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:53.609997 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:53.610336 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:53.610835 (+ 499us) server_negotiation.cc:299] Negotiation successful
0928 16:36:53.611307 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":655,"thread_start_us":375,"threads_started":1}
I20260928 16:36:53.612989 13746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0: No bootstrap required, opened a new log
I20260928 16:36:53.622016 13746 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6597ff40bb4b76858c8616adbefbb0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42053 } }
I20260928 16:36:53.622534 13746 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:53.622646 13746 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6597ff40bb4b76858c8616adbefbb0, State: Initialized, Role: FOLLOWER
I20260928 16:36:53.623353 13746 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6597ff40bb4b76858c8616adbefbb0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42053 } }
I20260928 16:36:53.623613 13746 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:36:53.623762 13746 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:36:53.623941 13746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:53.626748 13746 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6597ff40bb4b76858c8616adbefbb0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42053 } }
I20260928 16:36:53.627434 13746 leader_election.cc:302] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc6597ff40bb4b76858c8616adbefbb0; no voters:
I20260928 16:36:53.628137 13746 leader_election.cc:288] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:36:53.628273 13751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:53.629165 13751 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [term 1 LEADER]: Becoming Leader. State: Replica: bc6597ff40bb4b76858c8616adbefbb0, State: Running, Role: LEADER
I20260928 16:36:53.629760 13751 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6597ff40bb4b76858c8616adbefbb0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42053 } }
I20260928 16:36:53.630860 13746 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:36:53.633167 13753 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc6597ff40bb4b76858c8616adbefbb0. Latest consensus state: current_term: 1 leader_uuid: "bc6597ff40bb4b76858c8616adbefbb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6597ff40bb4b76858c8616adbefbb0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42053 } } }
I20260928 16:36:53.633616 13753 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:53.633119 13752 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc6597ff40bb4b76858c8616adbefbb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc6597ff40bb4b76858c8616adbefbb0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42053 } } }
I20260928 16:36:53.633791 13752 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0 [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:53.636847 13758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:36:53.642313 13758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:36:53.653088 13758 catalog_manager.cc:1347] Generated new cluster ID: d91daffb4f984850a431e9dbe6bd8c30
I20260928 16:36:53.653247 13758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:36:53.666457 13758 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:36:53.668682 13758 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:36:53.682018 13758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc6597ff40bb4b76858c8616adbefbb0: Generated new TSK 0
I20260928 16:36:53.683130 13758 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:36:53.746695 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42053
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:54.134428 13772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:54.134786 13772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:54.134918 13772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:54.142894 13772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:54.143035 13772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:54.143194 13772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:36:54.154139 13772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:54.155889 13772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:54.157610 13772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:54.173118 13777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:54.173130 13778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:54.173168 13780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:54.173673 13772 server_base.cc:1034] running on GCE node
I20260928 16:36:54.174744 13772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:54.176018 13772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:54.177253 13772 hybrid_clock.cc:648] HybridClock initialized: now 1790613414177170 us; error 71 us; skew 500 ppm
I20260928 16:36:54.177632 13772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:54 dist-test-slave-9fn6 krb5kdc[13665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:54.181470 13772 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:36:54.183389 13772 webserver.cc:460] Webserver started at http://127.10.247.193:34045/ using document root <none> and password file <none>
I20260928 16:36:54.184311 13772 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:54.184453 13772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:54.184856 13772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:54.188457 13772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "578ed5ceacc6413885b29a611ec1dc86"
format_stamp: "Formatted at 2026-09-28 16:36:54 on dist-test-slave-9fn6"
server_key: "b658313da66c44d264a4b4339b374668"
server_key_iv: "736ad3d4d01c6df955041d92fe25cd15"
server_key_version: "encryptionkey@0"
I20260928 16:36:54.189381 13772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "578ed5ceacc6413885b29a611ec1dc86"
format_stamp: "Formatted at 2026-09-28 16:36:54 on dist-test-slave-9fn6"
server_key: "b658313da66c44d264a4b4339b374668"
server_key_iv: "736ad3d4d01c6df955041d92fe25cd15"
server_key_version: "encryptionkey@0"
I20260928 16:36:54.195497 13772 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:36:54.199790 13787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:54.201478 13772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:36:54.201674 13772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "578ed5ceacc6413885b29a611ec1dc86"
format_stamp: "Formatted at 2026-09-28 16:36:54 on dist-test-slave-9fn6"
server_key: "b658313da66c44d264a4b4339b374668"
server_key_iv: "736ad3d4d01c6df955041d92fe25cd15"
server_key_version: "encryptionkey@0"
I20260928 16:36:54.201905 13772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:54.224767 13772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:54.229789 13772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:54.230065 13772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:54.231204 13772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:36:54.232849 13772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:54.232935 13772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:54.233027 13772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:54.233072 13772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:54.264647 13772 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:36571
I20260928 16:36:54.264701 13900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:36571 every 8 connection(s)
I20260928 16:36:54.266249 13772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:36:54.274047 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 13772
I20260928 16:36:54.274324 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:36:54.274986 11231 external_mini_cluster.cc:1428] Setting key 9c721b178c466ef84e8e9e19b11d6c42
Sep 28 16:36:54 dist-test-slave-9fn6 krb5kdc[13665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:54.289178 13903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:54.269517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:54729 (local address 127.10.247.254:42053)
0928 16:36:54.270088 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:54.270094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:54.271193 (+ 1099us) server_negotiation.cc:407] Connection header received
0928 16:36:54.273348 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:54.273354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:54.273449 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:54.273581 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:54.275570 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:54.276444 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:54.278230 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:54.278673 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:54.278800 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:54.282805 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:54.282811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:54.282814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:54.282852 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:54.285766 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:54.286555 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:54.286560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:54.286562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:54.286695 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:54.287155 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:54.287159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:54.287161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:54.287473 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:54.287628 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:54.288449 (+ 821us) server_negotiation.cc:299] Negotiation successful
0928 16:36:54.288764 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":287,"threads_started":1}
I20260928 16:36:54.290537 13904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:54.270172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42053 (local address 127.10.247.193:54729)
0928 16:36:54.270989 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:54.271062 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:54.273037 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:54.273870 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:54.273883 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:54.274613 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:54.275354 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:54.275376 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:54.276686 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:54.276691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:54.278078 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:54.278091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:54.278930 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:54.278937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:54.279215 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:54.279963 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:54.280037 (+ 74us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:54.282549 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:54.285907 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:54.285917 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:54.285940 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:54.286415 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:54.286805 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:54.286810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:54.286814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:54.287033 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:54.287605 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:54.287614 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:54.288034 (+ 420us) client_negotiation.cc:769] Sending connection context
0928 16:36:54.288309 (+ 275us) client_negotiation.cc:240] Negotiation successful
0928 16:36:54.288970 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":291,"threads_started":1}
I20260928 16:36:54.292346 13901 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42053
I20260928 16:36:54.292793 13901 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:54.293911 13901 heartbeater.cc:511] Master 127.10.247.254:42053 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
I20260928 16:36:54.296734 13711 ts_manager.cc:194] Registered new tserver with Master: 578ed5ceacc6413885b29a611ec1dc86 (127.10.247.193:36571)
I20260928 16:36:54.298405 13711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:54729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:36:54.324306 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42053
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:54.694233 13907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:54.694587 13907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:54.694746 13907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:54.702687 13907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:54.702852 13907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:54.703027 13907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:36:54.714279 13907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:54.716097 13907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:54.717792 13907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:54.732988 13913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:54.733062 13912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:54.733366 13915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:54.733820 13907 server_base.cc:1034] running on GCE node
I20260928 16:36:54.734529 13907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:54.735674 13907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:54.736917 13907 hybrid_clock.cc:648] HybridClock initialized: now 1790613414736842 us; error 56 us; skew 500 ppm
I20260928 16:36:54.737329 13907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:54 dist-test-slave-9fn6 krb5kdc[13665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:54.741148 13907 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:36:54.743153 13907 webserver.cc:460] Webserver started at http://127.10.247.194:34379/ using document root <none> and password file <none>
I20260928 16:36:54.744124 13907 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:54.744277 13907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:54.744709 13907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:54.748859 13907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "61dc374da1bd471dbfbdc58ee1e43a3a"
format_stamp: "Formatted at 2026-09-28 16:36:54 on dist-test-slave-9fn6"
server_key: "8832d5b807cc2574f2731761028bdf55"
server_key_iv: "0a07ae780c48e71ad2d593e87f844f7f"
server_key_version: "encryptionkey@0"
I20260928 16:36:54.750000 13907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "61dc374da1bd471dbfbdc58ee1e43a3a"
format_stamp: "Formatted at 2026-09-28 16:36:54 on dist-test-slave-9fn6"
server_key: "8832d5b807cc2574f2731761028bdf55"
server_key_iv: "0a07ae780c48e71ad2d593e87f844f7f"
server_key_version: "encryptionkey@0"
I20260928 16:36:54.757128 13907 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260928 16:36:54.761679 13922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:54.763468 13907 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:36:54.763674 13907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "61dc374da1bd471dbfbdc58ee1e43a3a"
format_stamp: "Formatted at 2026-09-28 16:36:54 on dist-test-slave-9fn6"
server_key: "8832d5b807cc2574f2731761028bdf55"
server_key_iv: "0a07ae780c48e71ad2d593e87f844f7f"
server_key_version: "encryptionkey@0"
I20260928 16:36:54.763919 13907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:54.783670 13907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:54.788681 13907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:54.789017 13907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:54.790212 13907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:36:54.791968 13907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:54.792055 13907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:54.792192 13907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:54.792248 13907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:54.826828 13907 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:34261
I20260928 16:36:54.826864 14035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:34261 every 8 connection(s)
I20260928 16:36:54.828434 13907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:36:54.837925 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 13907
I20260928 16:36:54.838145 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:36:54.838660 11231 external_mini_cluster.cc:1428] Setting key a218ff922de60f5ed8593d4b28a1f57f
Sep 28 16:36:54 dist-test-slave-9fn6 krb5kdc[13665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:54.852756 14038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:54.831604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:48347 (local address 127.10.247.254:42053)
0928 16:36:54.832179 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:54.832187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:54.833409 (+ 1222us) server_negotiation.cc:407] Connection header received
0928 16:36:54.835474 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:54.835479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:54.835570 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:54.835678 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:54.837652 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:54.838555 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:54.840460 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:54.841082 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:54.841187 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:54.846115 (+ 4928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:54.846123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:54.846128 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:54.846169 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:54.849187 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:54.850030 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:54.850036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:54.850039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:54.850215 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:54.850671 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:54.850676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:54.850679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:54.851073 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:54.851226 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:54.852019 (+ 793us) server_negotiation.cc:299] Negotiation successful
0928 16:36:54.852306 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":258,"threads_started":1}
I20260928 16:36:54.853914 14039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:54.832359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42053 (local address 127.10.247.194:48347)
0928 16:36:54.833223 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:54.833267 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:54.835166 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:54.835961 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:54.835971 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:54.836697 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:54.837449 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:54.837471 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:54.838726 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:54.838731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:54.840286 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:54.840300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:54.841315 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:54.841323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:54.841644 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:54.842519 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:54.842562 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:54.845792 (+ 3230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:54.849345 (+ 3553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:54.849354 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:54.849373 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:54.849887 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:54.850333 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:54.850336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:54.850339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:54.850563 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:54.851198 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:54.851206 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:54.851638 (+ 432us) client_negotiation.cc:769] Sending connection context
0928 16:36:54.851898 (+ 260us) client_negotiation.cc:240] Negotiation successful
0928 16:36:54.852414 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":333,"threads_started":1}
I20260928 16:36:54.855737 14036 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42053
I20260928 16:36:54.856113 14036 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:54.857189 14036 heartbeater.cc:511] Master 127.10.247.254:42053 requested a full tablet report, sending...
I20260928 16:36:54.859179 13711 ts_manager.cc:194] Registered new tserver with Master: 61dc374da1bd471dbfbdc58ee1e43a3a (127.10.247.194:34261)
I20260928 16:36:54.860148 13711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:48347
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:36:54.892318 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42053
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:55.277788 14042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:55.278172 14042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:55.278316 14042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:55.286346 14042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:55.286516 14042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:55.286691 14042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:36:55.297821 14042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:55.299590 14042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:55.301355 14042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 16:36:55.302260 13901 heartbeater.cc:503] Master 127.10.247.254:42053 was elected leader, sending a full tablet report...
W20260928 16:36:55.316496 14047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:55.317112 14048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:55.318233 14050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:55.319144 14042 server_base.cc:1034] running on GCE node
I20260928 16:36:55.319840 14042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:55.320977 14042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:55.322225 14042 hybrid_clock.cc:648] HybridClock initialized: now 1790613415322175 us; error 51 us; skew 500 ppm
I20260928 16:36:55.322571 14042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:55 dist-test-slave-9fn6 krb5kdc[13665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:55.326102 14042 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:36:55.327953 14042 webserver.cc:460] Webserver started at http://127.10.247.195:33743/ using document root <none> and password file <none>
I20260928 16:36:55.328928 14042 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:55.329069 14042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:55.329428 14042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:55.333195 14042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "5830fa3af2e04d1ca7421840809b27a7"
format_stamp: "Formatted at 2026-09-28 16:36:55 on dist-test-slave-9fn6"
server_key: "3e20c27e26b0d0f1db18198732ae1aec"
server_key_iv: "4f529af310e908fdb2f6085e126cb6b8"
server_key_version: "encryptionkey@0"
I20260928 16:36:55.334070 14042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "5830fa3af2e04d1ca7421840809b27a7"
format_stamp: "Formatted at 2026-09-28 16:36:55 on dist-test-slave-9fn6"
server_key: "3e20c27e26b0d0f1db18198732ae1aec"
server_key_iv: "4f529af310e908fdb2f6085e126cb6b8"
server_key_version: "encryptionkey@0"
I20260928 16:36:55.340155 14042 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260928 16:36:55.344180 14057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:55.345738 14042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:36:55.345884 14042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "5830fa3af2e04d1ca7421840809b27a7"
format_stamp: "Formatted at 2026-09-28 16:36:55 on dist-test-slave-9fn6"
server_key: "3e20c27e26b0d0f1db18198732ae1aec"
server_key_iv: "4f529af310e908fdb2f6085e126cb6b8"
server_key_version: "encryptionkey@0"
I20260928 16:36:55.346081 14042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:55.361260 14042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:55.365872 14042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:55.366168 14042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:55.367295 14042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:36:55.368966 14042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:36:55.369064 14042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:55.369215 14042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:36:55.369280 14042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:55.403879 14042 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:36239
I20260928 16:36:55.403937 14170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:36239 every 8 connection(s)
I20260928 16:36:55.405498 14042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:36:55.409462 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 14042
I20260928 16:36:55.409685 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:36:55.410332 11231 external_mini_cluster.cc:1428] Setting key 140ae8540c9afadbf13233ad188430c6
Sep 28 16:36:55 dist-test-slave-9fn6 krb5kdc[13665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:55.428396 14173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:55.408596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:46873 (local address 127.10.247.254:42053)
0928 16:36:55.409288 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:55.409296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:55.410409 (+ 1113us) server_negotiation.cc:407] Connection header received
0928 16:36:55.413009 (+ 2600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:55.413013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:55.413089 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:55.413185 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:55.415108 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:55.416075 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:55.417882 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:55.418322 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:55.418426 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:55.422236 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:55.422241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:55.422244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:55.422285 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:55.425027 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:55.425851 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:55.425858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:55.425861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:55.426024 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:55.426516 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:55.426522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:55.426524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:55.426875 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:55.427009 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:55.427794 (+ 785us) server_negotiation.cc:299] Negotiation successful
0928 16:36:55.427993 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":268,"threads_started":1}
I20260928 16:36:55.429548 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:55.409295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42053 (local address 127.10.247.195:46873)
0928 16:36:55.410211 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0928 16:36:55.410267 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:36:55.412704 (+ 2437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:36:55.413475 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:36:55.413487 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:36:55.414204 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:36:55.414917 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:55.414936 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:55.416259 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:55.416264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:55.417742 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:36:55.417756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:36:55.418519 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:36:55.418525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:36:55.418751 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:55.419513 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:36:55.419540 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:36:55.421989 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:36:55.425175 (+ 3186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:55.425185 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:55.425203 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:55.425725 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:55.426145 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:36:55.426150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:36:55.426154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:36:55.426376 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:36:55.427001 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:36:55.427009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:36:55.427426 (+ 417us) client_negotiation.cc:769] Sending connection context
0928 16:36:55.427680 (+ 254us) client_negotiation.cc:240] Negotiation successful
0928 16:36:55.428155 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":351,"threads_started":1}
I20260928 16:36:55.430912 14171 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42053
I20260928 16:36:55.431274 14171 heartbeater.cc:464] Registering TS with master...
I20260928 16:36:55.432156 14171 heartbeater.cc:511] Master 127.10.247.254:42053 requested a full tablet report, sending...
I20260928 16:36:55.433856 13711 ts_manager.cc:194] Registered new tserver with Master: 5830fa3af2e04d1ca7421840809b27a7 (127.10.247.195:36239)
I20260928 16:36:55.434849 13711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:46873
I20260928 16:36:55.439581 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 16:36:55.478068 14173 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:36:55.475044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.10.247.254:42053)
0928 16:36:55.475267 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:55.475273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:55.475667 (+ 394us) server_negotiation.cc:407] Connection header received
0928 16:36:55.475905 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:55.475910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:55.476045 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:55.476159 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:55.477740 (+ 1581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39860: BlockingRecv error: recv got EOF from 127.0.0.1:39860 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:36:55.481426 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13772
W20260928 16:36:55.485157 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13772: Not authorized: Client connection negotiation failed: client connection to 127.10.247.193:36571: server requires authentication, but client does not have Kerberos credentials available
I20260928 16:36:55.485406 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 13772
I20260928 16:36:55.501586 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13907
W20260928 16:36:55.504951 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13907: Not authorized: Client connection negotiation failed: client connection to 127.10.247.194:34261: server requires authentication, but client does not have Kerberos credentials available
I20260928 16:36:55.505215 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 13907
I20260928 16:36:55.521147 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14042
W20260928 16:36:55.524833 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14042: Not authorized: Client connection negotiation failed: client connection to 127.10.247.195:36239: server requires authentication, but client does not have Kerberos credentials available
I20260928 16:36:55.525019 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 14042
I20260928 16:36:55.540819 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 13678
I20260928 16:36:55.542385 13741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:55.659118 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 13678
2026-09-28T16:36:55Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3830 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:36:55 dist-test-slave-9fn6 krb5kdc[14189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:36:55 dist-test-slave-9fn6 krb5kdc[14189](info): set up 2 sockets
Sep 28 16:36:55 dist-test-slave-9fn6 krb5kdc[14189](info): commencing operation
krb5kdc: starting...
W20260928 16:36:56.685334 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:36:56 dist-test-slave-9fn6 krb5kdc[14189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:36:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:36:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:36:56.912396 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:39873
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:38765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:39873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:57.293004 14202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:57.293413 14202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:57.293567 14202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:57.301580 14202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:36:57.301743 14202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:57.301831 14202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:57.301914 14202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:36:57.301992 14202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:36:57.313337 14202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:38765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:39873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:39873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:57.315114 14202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:57.316807 14202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:57.330062 14207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:57.330101 14210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:57.330039 14208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:57.330917 14202 server_base.cc:1034] running on GCE node
I20260928 16:36:57.331844 14202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:57.333635 14202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:57.334865 14202 hybrid_clock.cc:648] HybridClock initialized: now 1790613417334794 us; error 67 us; skew 500 ppm
I20260928 16:36:57.335343 14202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:57 dist-test-slave-9fn6 krb5kdc[14189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613417, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:57.344941 14202 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:36:57.346982 14202 webserver.cc:460] Webserver started at http://127.10.247.254:43115/ using document root <none> and password file <none>
I20260928 16:36:57.347918 14202 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:57.348054 14202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:57.348433 14202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:57.352245 14202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "ccbe4d4174b74f00b23073a346d8b277"
format_stamp: "Formatted at 2026-09-28 16:36:57 on dist-test-slave-9fn6"
server_key: "23fde379fa440e9fe6c256eea03243a5"
server_key_iv: "e9cdfc74652f00703b1bcb7c05230194"
server_key_version: "encryptionkey@0"
I20260928 16:36:57.353257 14202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "ccbe4d4174b74f00b23073a346d8b277"
format_stamp: "Formatted at 2026-09-28 16:36:57 on dist-test-slave-9fn6"
server_key: "23fde379fa440e9fe6c256eea03243a5"
server_key_iv: "e9cdfc74652f00703b1bcb7c05230194"
server_key_version: "encryptionkey@0"
I20260928 16:36:57.359782 14202 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260928 16:36:57.364107 14217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:57.365806 14202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:36:57.365994 14202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "ccbe4d4174b74f00b23073a346d8b277"
format_stamp: "Formatted at 2026-09-28 16:36:57 on dist-test-slave-9fn6"
server_key: "23fde379fa440e9fe6c256eea03243a5"
server_key_iv: "e9cdfc74652f00703b1bcb7c05230194"
server_key_version: "encryptionkey@0"
I20260928 16:36:57.366264 14202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:57.391140 14202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:57.396190 14202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:57.396498 14202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:57.416250 14202 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:39873
I20260928 16:36:57.416253 14268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:39873 every 8 connection(s)
I20260928 16:36:57.418049 14202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:36:57.422700 14269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:36:57.426757 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 14202
I20260928 16:36:57.426999 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:36:57.427722 11231 external_mini_cluster.cc:1428] Setting key 09d7c953d06e24b5cce87cc48a18698f
I20260928 16:36:57.436833 14269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277: Bootstrap starting.
I20260928 16:36:57.441514 14269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:57.443161 14269 log.cc:824] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277: Log is configured to *not* fsync() on all Append() calls
Sep 28 16:36:57 dist-test-slave-9fn6 krb5kdc[14189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:57.446947 14269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277: No bootstrap required, opened a new log
I20260928 16:36:57.453562 14272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:57.429913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43934 (local address 127.10.247.254:39873)
0928 16:36:57.430820 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:57.430836 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:57.430884 (+ 48us) server_negotiation.cc:407] Connection header received
0928 16:36:57.432751 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:57.432782 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:57.433285 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:57.433711 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:57.435415 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:57.436935 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:57.438795 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:57.439577 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:57.439729 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:57.445172 (+ 5443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:57.445182 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:57.445203 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:57.445247 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:57.448518 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:57.449390 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:57.449398 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:57.449410 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:57.449612 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:57.450047 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:57.450058 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:57.450060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:57.450771 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:57.451105 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:57.451669 (+ 564us) server_negotiation.cc:299] Negotiation successful
0928 16:36:57.452202 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":380,"threads_started":1}
I20260928 16:36:57.456166 14269 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe4d4174b74f00b23073a346d8b277" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39873 } }
I20260928 16:36:57.456733 14269 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:57.456849 14269 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbe4d4174b74f00b23073a346d8b277, State: Initialized, Role: FOLLOWER
I20260928 16:36:57.457546 14269 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe4d4174b74f00b23073a346d8b277" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39873 } }
I20260928 16:36:57.457821 14269 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:36:57.457965 14269 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:36:57.458160 14269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:57.458354 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:36:57.460681 14269 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe4d4174b74f00b23073a346d8b277" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39873 } }
I20260928 16:36:57.461315 14269 leader_election.cc:302] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccbe4d4174b74f00b23073a346d8b277; no voters:
I20260928 16:36:57.462080 14269 leader_election.cc:288] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:36:57.462214 14274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:57.463419 14274 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [term 1 LEADER]: Becoming Leader. State: Replica: ccbe4d4174b74f00b23073a346d8b277, State: Running, Role: LEADER
I20260928 16:36:57.464125 14274 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe4d4174b74f00b23073a346d8b277" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39873 } }
I20260928 16:36:57.464876 14269 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:36:57.467813 14275 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccbe4d4174b74f00b23073a346d8b277" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe4d4174b74f00b23073a346d8b277" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39873 } } }
I20260928 16:36:57.468376 14275 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:57.467969 14276 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccbe4d4174b74f00b23073a346d8b277. Latest consensus state: current_term: 1 leader_uuid: "ccbe4d4174b74f00b23073a346d8b277" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe4d4174b74f00b23073a346d8b277" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39873 } } }
I20260928 16:36:57.472633 14276 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277 [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:57.476912 14280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:36:57.484063 14280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:36:57.499357 14280 catalog_manager.cc:1347] Generated new cluster ID: f76a4d1255a94e938076771e20796e12
I20260928 16:36:57.499637 14280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:36:57.512153 14295 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 16:36:57.513895 14280 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:36:57.515460 14280 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:36:57.526608 14280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccbe4d4174b74f00b23073a346d8b277: Generated new TSK 0
I20260928 16:36:57.527778 14280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:36:57.529275 14272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:57.518694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.10.247.254:39873)
0928 16:36:57.518883 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:57.518892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:57.519429 (+ 537us) server_negotiation.cc:407] Connection header received
0928 16:36:57.519670 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:57.519676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:57.519767 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:57.519874 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:57.521373 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:57.522536 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:57.524707 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:57.525451 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:57.525594 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:57.527122 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:57.527130 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:57.527135 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:36:57.528270 (+ 1135us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:57.528373 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:36:57.528414 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:57.528557 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:57.528748 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 16:36:57.528881 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 16:36:57.538419 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14202
I20260928 16:36:57.539800 14264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:36:57.637809 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 14202
2026-09-28T16:36:57Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1973 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:36:57 dist-test-slave-9fn6 krb5kdc[14303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:36:57 dist-test-slave-9fn6 krb5kdc[14303](info): set up 2 sockets
Sep 28 16:36:57 dist-test-slave-9fn6 krb5kdc[14303](info): commencing operation
krb5kdc: starting...
W20260928 16:36:58.677434 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:36:58 dist-test-slave-9fn6 krb5kdc[14303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:36:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:36:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:36:58.924686 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44885
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:44885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:59.307088 14316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:59.307474 14316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:59.307574 14316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:59.315702 14316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:36:59.315848 14316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:59.315898 14316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:59.315964 14316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:36:59.316017 14316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:36:59.327083 14316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:44885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:36:59.328856 14316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:36:59.330471 14316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:36:59.344053 14321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:59.344087 14324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:36:59.343990 14322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:36:59.344875 14316 server_base.cc:1034] running on GCE node
I20260928 16:36:59.345856 14316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:36:59.347633 14316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:36:59.348863 14316 hybrid_clock.cc:648] HybridClock initialized: now 1790613419348805 us; error 48 us; skew 500 ppm
I20260928 16:36:59.349269 14316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:36:59 dist-test-slave-9fn6 krb5kdc[14303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613419, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:36:59.353085 14316 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:36:59.354961 14316 webserver.cc:460] Webserver started at http://127.10.247.254:38867/ using document root <none> and password file <none>
I20260928 16:36:59.355834 14316 fs_manager.cc:359] Metadata directory not provided
I20260928 16:36:59.355967 14316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:36:59.356328 14316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:36:59.360184 14316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "f9c88bad1be146c1b503204935a1f93b"
format_stamp: "Formatted at 2026-09-28 16:36:59 on dist-test-slave-9fn6"
server_key: "36ccf96c916ef88e0d11f87640501948"
server_key_iv: "cbcef7693194d1f87cc531b69257b097"
server_key_version: "encryptionkey@0"
I20260928 16:36:59.361159 14316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "f9c88bad1be146c1b503204935a1f93b"
format_stamp: "Formatted at 2026-09-28 16:36:59 on dist-test-slave-9fn6"
server_key: "36ccf96c916ef88e0d11f87640501948"
server_key_iv: "cbcef7693194d1f87cc531b69257b097"
server_key_version: "encryptionkey@0"
I20260928 16:36:59.367744 14316 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:36:59.372428 14331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:36:59.374251 14316 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:36:59.374511 14316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "f9c88bad1be146c1b503204935a1f93b"
format_stamp: "Formatted at 2026-09-28 16:36:59 on dist-test-slave-9fn6"
server_key: "36ccf96c916ef88e0d11f87640501948"
server_key_iv: "cbcef7693194d1f87cc531b69257b097"
server_key_version: "encryptionkey@0"
I20260928 16:36:59.374771 14316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:36:59.390573 14316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:36:59.395977 14316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:36:59.396341 14316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:36:59.417116 14316 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:44885
I20260928 16:36:59.417162 14382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:44885 every 8 connection(s)
I20260928 16:36:59.418864 14316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:36:59.420315 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 14316
I20260928 16:36:59.420580 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:36:59.421490 11231 external_mini_cluster.cc:1428] Setting key 1ce6d346bb44d2a4273bd25c6a7a3362
I20260928 16:36:59.424801 14383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 16:36:59 dist-test-slave-9fn6 krb5kdc[14303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:36:59.439719 14383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b: Bootstrap starting.
I20260928 16:36:59.443742 14383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b: Neither blocks nor log segments found. Creating new log.
I20260928 16:36:59.445010 14383 log.cc:824] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:36:59.447747 14386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:36:59.424060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.10.247.254:44885)
0928 16:36:59.425017 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0928 16:36:59.425041 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0928 16:36:59.425095 (+ 54us) server_negotiation.cc:407] Connection header received
0928 16:36:59.426901 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:36:59.426927 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:36:59.427362 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:36:59.427761 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:36:59.429765 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:59.431266 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:59.433175 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:36:59.433940 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:36:59.434101 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:36:59.439400 (+ 5299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:36:59.439426 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:36:59.439444 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:36:59.439494 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:36:59.442655 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:59.443565 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:59.443572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:59.443579 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:59.443790 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:36:59.444308 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:36:59.444313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:36:59.444315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:36:59.445096 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:36:59.445430 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0928 16:36:59.445968 (+ 538us) server_negotiation.cc:299] Negotiation successful
0928 16:36:59.446530 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":408,"threads_started":1}
I20260928 16:36:59.448639 14383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b: No bootstrap required, opened a new log
I20260928 16:36:59.457249 14383 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c88bad1be146c1b503204935a1f93b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44885 } }
I20260928 16:36:59.457701 14383 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:36:59.457800 14383 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9c88bad1be146c1b503204935a1f93b, State: Initialized, Role: FOLLOWER
I20260928 16:36:59.458411 14383 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c88bad1be146c1b503204935a1f93b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44885 } }
I20260928 16:36:59.458688 14383 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:36:59.458837 14383 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:36:59.459048 14383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:36:59.461479 14383 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c88bad1be146c1b503204935a1f93b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44885 } }
I20260928 16:36:59.462045 14383 leader_election.cc:302] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9c88bad1be146c1b503204935a1f93b; no voters:
I20260928 16:36:59.462697 14383 leader_election.cc:288] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:36:59.462881 14388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:36:59.464097 14388 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [term 1 LEADER]: Becoming Leader. State: Replica: f9c88bad1be146c1b503204935a1f93b, State: Running, Role: LEADER
I20260928 16:36:59.464699 14388 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c88bad1be146c1b503204935a1f93b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44885 } }
I20260928 16:36:59.465330 14383 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:36:59.467672 14389 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9c88bad1be146c1b503204935a1f93b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c88bad1be146c1b503204935a1f93b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44885 } } }
I20260928 16:36:59.467656 14390 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9c88bad1be146c1b503204935a1f93b. Latest consensus state: current_term: 1 leader_uuid: "f9c88bad1be146c1b503204935a1f93b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c88bad1be146c1b503204935a1f93b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44885 } } }
I20260928 16:36:59.468110 14390 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:59.468110 14389 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b [sys.catalog]: This master's current role is: LEADER
I20260928 16:36:59.469151 14395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:36:59.475406 14395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:36:59.485859 14395 catalog_manager.cc:1347] Generated new cluster ID: 56c340d953034519be7a12b4340ca973
I20260928 16:36:59.486006 14395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:36:59.498844 14395 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:36:59.500379 14395 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:36:59.508159 14395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9c88bad1be146c1b503204935a1f93b: Generated new TSK 0
I20260928 16:36:59.509414 14395 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:36:59.582765 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44885
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:36:59.979773 14409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:36:59.980168 14409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:36:59.980312 14409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:36:59.988170 14409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:36:59.988320 14409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:36:59.988492 14409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:36:59.999507 14409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:00.001469 14409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:00.003269 14409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:00.019032 14415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:00.019012 14417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:00.018959 14414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:00.020046 14409 server_base.cc:1034] running on GCE node
I20260928 16:37:00.020869 14409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:00.022037 14409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:00.023285 14409 hybrid_clock.cc:648] HybridClock initialized: now 1790613420023226 us; error 42 us; skew 500 ppm
I20260928 16:37:00.023670 14409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:00 dist-test-slave-9fn6 krb5kdc[14303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:00.027355 14409 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:37:00.029147 14409 webserver.cc:460] Webserver started at http://127.10.247.193:42259/ using document root <none> and password file <none>
I20260928 16:37:00.030043 14409 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:00.030177 14409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:00.030539 14409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:00.034312 14409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "1f247c5abcff4ca983ded9f0f2e57d7e"
format_stamp: "Formatted at 2026-09-28 16:37:00 on dist-test-slave-9fn6"
server_key: "b4cf2da4702c6db866a9669a068f6d63"
server_key_iv: "2e0940861a13ffa9b89745cda8ef29d9"
server_key_version: "encryptionkey@0"
I20260928 16:37:00.035215 14409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "1f247c5abcff4ca983ded9f0f2e57d7e"
format_stamp: "Formatted at 2026-09-28 16:37:00 on dist-test-slave-9fn6"
server_key: "b4cf2da4702c6db866a9669a068f6d63"
server_key_iv: "2e0940861a13ffa9b89745cda8ef29d9"
server_key_version: "encryptionkey@0"
I20260928 16:37:00.041461 14409 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260928 16:37:00.045534 14424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:00.047128 14409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:37:00.047302 14409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "1f247c5abcff4ca983ded9f0f2e57d7e"
format_stamp: "Formatted at 2026-09-28 16:37:00 on dist-test-slave-9fn6"
server_key: "b4cf2da4702c6db866a9669a068f6d63"
server_key_iv: "2e0940861a13ffa9b89745cda8ef29d9"
server_key_version: "encryptionkey@0"
I20260928 16:37:00.047521 14409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:00.067358 14409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:00.072042 14409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:00.072374 14409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:00.073587 14409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:00.075280 14409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:00.075374 14409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:00.075522 14409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:00.075578 14409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:00.110785 14409 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:44359
I20260928 16:37:00.110808 14536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:44359 every 8 connection(s)
I20260928 16:37:00.112375 14409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:00.122318 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 14409
I20260928 16:37:00.122584 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:37:00.123580 11231 external_mini_cluster.cc:1428] Setting key 9ee5078e5a0647924c834cb02ca54749
Sep 28 16:37:00 dist-test-slave-9fn6 krb5kdc[14303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:00.137604 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:00.115654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:57845 (local address 127.10.247.254:44885)
0928 16:37:00.116247 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:00.116254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:00.117658 (+ 1404us) server_negotiation.cc:407] Connection header received
0928 16:37:00.119791 (+ 2133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:00.119796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:00.119887 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:00.120002 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:00.121906 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:00.123113 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:00.125180 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:00.125779 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:00.125917 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:00.130716 (+ 4799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:00.130723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:00.130727 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:00.130774 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:00.133914 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:00.134747 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:00.134753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:00.134757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:00.134935 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:00.135380 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:00.135385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:00.135389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:00.135799 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:00.135956 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:00.136878 (+ 922us) server_negotiation.cc:299] Negotiation successful
0928 16:37:00.137190 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":260,"threads_started":1}
I20260928 16:37:00.138768 14541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:00.116483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44885 (local address 127.10.247.193:57845)
0928 16:37:00.117438 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:00.117494 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:00.119459 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:00.120259 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:00.120270 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:00.120955 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:00.121687 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:00.121707 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:00.123306 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:00.123310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:00.125003 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:00.125016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:00.126034 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:00.126040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:00.126354 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:00.127198 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:00.127245 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:00.130362 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:00.134073 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:00.134081 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:00.134101 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:00.134601 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:00.135039 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:00.135043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:00.135045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:00.135274 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:00.135926 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:00.135933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:00.136394 (+ 461us) client_negotiation.cc:769] Sending connection context
0928 16:37:00.136739 (+ 345us) client_negotiation.cc:240] Negotiation successful
0928 16:37:00.137301 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":344,"threads_started":1}
I20260928 16:37:00.140833 14537 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44885
I20260928 16:37:00.141250 14537 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:00.142350 14537 heartbeater.cc:511] Master 127.10.247.254:44885 requested a full tablet report, sending...
I20260928 16:37:00.145179 14348 ts_manager.cc:194] Registered new tserver with Master: 1f247c5abcff4ca983ded9f0f2e57d7e (127.10.247.193:44359)
I20260928 16:37:00.146914 14348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:57845
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:37:00.176327 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44885
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:00.556356 14544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:00.556818 14544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:00.556948 14544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:00.564790 14544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:00.564908 14544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:00.565048 14544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:00.575955 14544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:00.577693 14544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:00.579324 14544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:00.597225 14550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:00.597246 14553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:00.597273 14551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:00.598119 14544 server_base.cc:1034] running on GCE node
I20260928 16:37:00.598867 14544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:00.600036 14544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:00.601269 14544 hybrid_clock.cc:648] HybridClock initialized: now 1790613420601213 us; error 50 us; skew 500 ppm
I20260928 16:37:00.601634 14544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:00 dist-test-slave-9fn6 krb5kdc[14303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:00.605165 14544 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:37:00.606804 14544 webserver.cc:460] Webserver started at http://127.10.247.194:44429/ using document root <none> and password file <none>
I20260928 16:37:00.607617 14544 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:00.607734 14544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:00.608081 14544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:00.611644 14544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "97e8650dc62b44448c780b9e03a50372"
format_stamp: "Formatted at 2026-09-28 16:37:00 on dist-test-slave-9fn6"
server_key: "fad4e0b82b27179e771034a0f0f06a9b"
server_key_iv: "8d4d493cfeb4a4e9305423a8b3cf5dfe"
server_key_version: "encryptionkey@0"
I20260928 16:37:00.612586 14544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "97e8650dc62b44448c780b9e03a50372"
format_stamp: "Formatted at 2026-09-28 16:37:00 on dist-test-slave-9fn6"
server_key: "fad4e0b82b27179e771034a0f0f06a9b"
server_key_iv: "8d4d493cfeb4a4e9305423a8b3cf5dfe"
server_key_version: "encryptionkey@0"
I20260928 16:37:00.619189 14544 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:37:00.623792 14560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:00.625705 14544 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260928 16:37:00.625908 14544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "97e8650dc62b44448c780b9e03a50372"
format_stamp: "Formatted at 2026-09-28 16:37:00 on dist-test-slave-9fn6"
server_key: "fad4e0b82b27179e771034a0f0f06a9b"
server_key_iv: "8d4d493cfeb4a4e9305423a8b3cf5dfe"
server_key_version: "encryptionkey@0"
I20260928 16:37:00.626205 14544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:00.655370 14544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:00.660343 14544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:00.660696 14544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:00.661926 14544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:00.663579 14544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:00.663658 14544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:00.663790 14544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:00.663842 14544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:00.697981 14544 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:38579
I20260928 16:37:00.698046 14672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:38579 every 8 connection(s)
I20260928 16:37:00.699527 14544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:00.702595 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 14544
I20260928 16:37:00.702811 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:37:00.703444 11231 external_mini_cluster.cc:1428] Setting key d0feca92010d3db45d3a1e8adada40b1
Sep 28 16:37:00 dist-test-slave-9fn6 krb5kdc[14303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:00.722631 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:00.702549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:36117 (local address 127.10.247.254:44885)
0928 16:37:00.703174 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:00.703180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:00.704443 (+ 1263us) server_negotiation.cc:407] Connection header received
0928 16:37:00.706690 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:00.706696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:00.706790 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:00.706893 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:00.708747 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:00.709574 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:00.711259 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:00.711719 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:00.711835 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:00.715928 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:00.715934 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:00.715938 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:00.715979 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:00.718906 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:00.719780 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:00.719784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:00.719787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:00.719945 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:00.720457 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:00.720461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:00.720463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:00.720861 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:00.721029 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:00.721893 (+ 864us) server_negotiation.cc:299] Negotiation successful
0928 16:37:00.722213 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":304,"threads_started":1}
I20260928 16:37:00.723901 14676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:00.703316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44885 (local address 127.10.247.194:36117)
0928 16:37:00.704206 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:00.704275 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:00.706392 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:00.707206 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:00.707217 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:00.707821 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:00.708527 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:00.708547 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:00.709730 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:00.709735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:00.711102 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:00.711115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:00.711962 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:00.711969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:00.712221 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:00.713015 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:00.713042 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:00.715643 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:00.719072 (+ 3429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:00.719083 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:00.719107 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:00.719635 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:00.720075 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:00.720080 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:00.720085 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:00.720319 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:00.721005 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:00.721015 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:00.721462 (+ 447us) client_negotiation.cc:769] Sending connection context
0928 16:37:00.721760 (+ 298us) client_negotiation.cc:240] Negotiation successful
0928 16:37:00.722458 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":349,"threads_started":1}
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
I20260928 16:37:00.725696 14673 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44885
I20260928 16:37:00.726101 14673 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
I20260928 16:37:00.727226 14673 heartbeater.cc:511] Master 127.10.247.254:44885 requested a full tablet report, sending...
I20260928 16:37:00.729395 14348 ts_manager.cc:194] Registered new tserver with Master: 97e8650dc62b44448c780b9e03a50372 (127.10.247.194:38579)
I20260928 16:37:00.730400 14348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:36117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:37:00.755486 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44885
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:01.143134 14679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:01.143486 14679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:01.143594 14679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260928 16:37:01.150784 14537 heartbeater.cc:503] Master 127.10.247.254:44885 was elected leader, sending a full tablet report...
W20260928 16:37:01.151687 14679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:01.151851 14679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:01.152050 14679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:01.163822 14679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:01.165745 14679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:01.167616 14679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:01.183903 14684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:01.183970 14687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:01.183879 14685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:01.184870 14679 server_base.cc:1034] running on GCE node
I20260928 16:37:01.185672 14679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:01.186914 14679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:01.188166 14679 hybrid_clock.cc:648] HybridClock initialized: now 1790613421188121 us; error 51 us; skew 500 ppm
I20260928 16:37:01.188566 14679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613421, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:01.192468 14679 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:37:01.194351 14679 webserver.cc:460] Webserver started at http://127.10.247.195:38623/ using document root <none> and password file <none>
I20260928 16:37:01.195240 14679 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:01.195376 14679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:01.195735 14679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:01.199574 14679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "184c734e14474e069c662579fea64b61"
format_stamp: "Formatted at 2026-09-28 16:37:01 on dist-test-slave-9fn6"
server_key: "cf37bf1944901a29f3e3adc8fbc41eea"
server_key_iv: "493ae32f8614235a761e279bda30f122"
server_key_version: "encryptionkey@0"
I20260928 16:37:01.200528 14679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "184c734e14474e069c662579fea64b61"
format_stamp: "Formatted at 2026-09-28 16:37:01 on dist-test-slave-9fn6"
server_key: "cf37bf1944901a29f3e3adc8fbc41eea"
server_key_iv: "493ae32f8614235a761e279bda30f122"
server_key_version: "encryptionkey@0"
I20260928 16:37:01.207365 14679 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:37:01.212180 14694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:01.214149 14679 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:01.214359 14679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "184c734e14474e069c662579fea64b61"
format_stamp: "Formatted at 2026-09-28 16:37:01 on dist-test-slave-9fn6"
server_key: "cf37bf1944901a29f3e3adc8fbc41eea"
server_key_iv: "493ae32f8614235a761e279bda30f122"
server_key_version: "encryptionkey@0"
I20260928 16:37:01.214615 14679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:01.233547 14679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:01.238703 14679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:01.239044 14679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:01.240312 14679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:01.242071 14679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:01.242161 14679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:01.242300 14679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:01.242354 14679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:01.277267 14679 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:39775
I20260928 16:37:01.277305 14806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:39775 every 8 connection(s)
I20260928 16:37:01.278792 14679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:01.280994 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 14679
I20260928 16:37:01.281209 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:37:01.281939 11231 external_mini_cluster.cc:1428] Setting key e51d95336eba3003d9c987e2d1ee34c0
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613421, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:01.302376 14809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:01.281844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:50521 (local address 127.10.247.254:44885)
0928 16:37:01.282392 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:01.282397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:01.283815 (+ 1418us) server_negotiation.cc:407] Connection header received
0928 16:37:01.286431 (+ 2616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:01.286437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:01.286545 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:01.286656 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:01.288474 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.289421 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.291146 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.291627 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.291766 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:01.295816 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:01.295821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:01.295824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:01.295865 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:01.299025 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.299796 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.299800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.299802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.299937 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.300445 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.300450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.300452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.300848 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:01.301012 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:01.301837 (+ 825us) server_negotiation.cc:299] Negotiation successful
0928 16:37:01.302042 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":274,"threads_started":1}
I20260928 16:37:01.303551 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:01.282618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44885 (local address 127.10.247.195:50521)
0928 16:37:01.283610 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:01.283676 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:01.286135 (+ 2459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:01.286949 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:01.286959 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:01.287572 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:01.288274 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:01.288296 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.289598 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.289602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:01.290990 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:01.291005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.291921 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.291931 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:01.292207 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:01.292964 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:01.292993 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:01.295563 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:01.299172 (+ 3609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:01.299181 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:01.299198 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:01.299690 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:01.300069 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:01.300073 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:01.300075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:01.300299 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:01.300971 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:01.300978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:01.301422 (+ 444us) client_negotiation.cc:769] Sending connection context
0928 16:37:01.301718 (+ 296us) client_negotiation.cc:240] Negotiation successful
0928 16:37:01.302203 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":378,"threads_started":1}
I20260928 16:37:01.305212 14807 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44885
I20260928 16:37:01.305545 14807 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:01.306396 14807 heartbeater.cc:511] Master 127.10.247.254:44885 requested a full tablet report, sending...
I20260928 16:37:01.308072 14348 ts_manager.cc:194] Registered new tserver with Master: 184c734e14474e069c662579fea64b61 (127.10.247.195:39775)
I20260928 16:37:01.309144 14348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:50521
I20260928 16:37:01.311447 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613421, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613421, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:01.367872 14809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:01.352449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.10.247.254:44885)
0928 16:37:01.352734 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:01.352743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:01.353267 (+ 524us) server_negotiation.cc:407] Connection header received
0928 16:37:01.353484 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:01.353489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:01.353575 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:01.353674 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:01.355238 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.356062 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.357391 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.357828 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.357958 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:01.362176 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:01.362181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:01.362185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:01.362231 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:01.364904 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.365683 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.365686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.365689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.365826 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.366308 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.366312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.366314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.366761 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:01.366983 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:01.367370 (+ 387us) server_negotiation.cc:299] Negotiation successful
0928 16:37:01.367574 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260928 16:37:01.368558 14348 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57332
I20260928 16:37:01.370747 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14409
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613421, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:37:01.388458 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:01.372007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.10.247.193:44359)
0928 16:37:01.372738 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:01.372748 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:01.372780 (+ 32us) server_negotiation.cc:407] Connection header received
0928 16:37:01.372848 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:01.372855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:01.373119 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:01.373316 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:01.374615 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.375714 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.377176 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.377838 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.377953 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:01.382262 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:01.382276 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:01.382295 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:01.382337 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:01.385434 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.386134 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.386146 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.386150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.386313 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.386757 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.386763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.386766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.387318 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:01.387589 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:01.387843 (+ 254us) server_negotiation.cc:299] Negotiation successful
0928 16:37:01.388126 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":350,"threads_started":1}
W20260928 16:37:01.390060 14532 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53198
W20260928 16:37:01.391353 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14409: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:37:01.391590 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 14409
I20260928 16:37:01.407389 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14544
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613421, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:37:01.426265 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:01.408474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34154 (local address 127.10.247.194:38579)
0928 16:37:01.409224 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:01.409233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:01.409263 (+ 30us) server_negotiation.cc:407] Connection header received
0928 16:37:01.409339 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:01.409348 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:01.409643 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:01.409792 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:01.411187 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.412308 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.413652 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.414282 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.414385 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:01.418891 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:01.418910 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:01.418936 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:01.418987 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:01.423076 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.423792 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.423803 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.423807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.423971 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.424327 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.424331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.424335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.424877 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:01.425219 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:01.425474 (+ 255us) server_negotiation.cc:299] Negotiation successful
0928 16:37:01.425806 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":392,"threads_started":1}
W20260928 16:37:01.427942 14668 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34154
W20260928 16:37:01.429402 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14544: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:37:01.429574 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 14544
I20260928 16:37:01.445554 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14679
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613421, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:37:01.464123 14821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:01.446705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.10.247.195:39775)
0928 16:37:01.447468 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:01.447477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:01.447511 (+ 34us) server_negotiation.cc:407] Connection header received
0928 16:37:01.447584 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:01.447592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:01.447889 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:01.448074 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:01.449748 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.450928 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.452297 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:01.453019 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:01.453146 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:01.457778 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:01.457792 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:01.457812 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:01.457857 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:01.460990 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.461758 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.461767 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.461770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.461915 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:01.462310 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:01.462315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:01.462317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:01.462740 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:01.463080 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:01.463395 (+ 315us) server_negotiation.cc:299] Negotiation successful
0928 16:37:01.463712 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":387,"threads_started":1}
W20260928 16:37:01.465729 14802 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41192
W20260928 16:37:01.467057 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14679: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:37:01.467242 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 14679
I20260928 16:37:01.482916 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14316
I20260928 16:37:01.484262 14378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:01.600942 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 14316
2026-09-28T16:37:01Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3969 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14825](info): set up 2 sockets
Sep 28 16:37:01 dist-test-slave-9fn6 krb5kdc[14825](info): commencing operation
krb5kdc: starting...
W20260928 16:37:02.621268 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.005s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:37:02 dist-test-slave-9fn6 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:37:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:37:02.849316 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:34961
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:34961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:03.222947 14838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:03.223325 14838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:03.223423 14838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:03.231099 14838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:03.231235 14838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:03.231289 14838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:03.231351 14838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:03.231397 14838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:03.242511 14838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:34961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:03.244273 14838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:03.246301 14838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:03.260308 14843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:03.260255 14844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:03.260285 14846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:03.261291 14838 server_base.cc:1034] running on GCE node
I20260928 16:37:03.262310 14838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:03.264101 14838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:03.265324 14838 hybrid_clock.cc:648] HybridClock initialized: now 1790613423265264 us; error 39 us; skew 500 ppm
I20260928 16:37:03.265740 14838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:03 dist-test-slave-9fn6 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613423, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:03.273594 14838 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:37:03.275555 14838 webserver.cc:460] Webserver started at http://127.10.247.254:36235/ using document root <none> and password file <none>
I20260928 16:37:03.276459 14838 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:03.276644 14838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:03.277035 14838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:03.280817 14838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "e5561c411e6540e5b09298ca5dcae967"
format_stamp: "Formatted at 2026-09-28 16:37:03 on dist-test-slave-9fn6"
server_key: "ed2620b10c440ff667d017a0328669ec"
server_key_iv: "a59c21fd26fd09220215f2d4e4a70205"
server_key_version: "encryptionkey@0"
I20260928 16:37:03.281749 14838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "e5561c411e6540e5b09298ca5dcae967"
format_stamp: "Formatted at 2026-09-28 16:37:03 on dist-test-slave-9fn6"
server_key: "ed2620b10c440ff667d017a0328669ec"
server_key_iv: "a59c21fd26fd09220215f2d4e4a70205"
server_key_version: "encryptionkey@0"
I20260928 16:37:03.287895 14838 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:37:03.292311 14853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:03.294061 14838 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:03.294275 14838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "e5561c411e6540e5b09298ca5dcae967"
format_stamp: "Formatted at 2026-09-28 16:37:03 on dist-test-slave-9fn6"
server_key: "ed2620b10c440ff667d017a0328669ec"
server_key_iv: "a59c21fd26fd09220215f2d4e4a70205"
server_key_version: "encryptionkey@0"
I20260928 16:37:03.294509 14838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:03.314771 14838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:03.319461 14838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:03.319772 14838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:03.339752 14838 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:34961
I20260928 16:37:03.339771 14904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:34961 every 8 connection(s)
I20260928 16:37:03.341420 14838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:03.343658 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 14838
I20260928 16:37:03.343889 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:03.344624 11231 external_mini_cluster.cc:1428] Setting key c70c0a9b266e25dc4dfa3d8a18ac43c6
I20260928 16:37:03.346135 14905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 16:37:03 dist-test-slave-9fn6 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:03.361718 14905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967: Bootstrap starting.
I20260928 16:37:03.365978 14905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:03.367268 14905 log.cc:824] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:03.370692 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:03.346833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.10.247.254:34961)
0928 16:37:03.347651 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:03.347664 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:03.347712 (+ 48us) server_negotiation.cc:407] Connection header received
0928 16:37:03.349564 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:03.349590 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:03.350058 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:03.350455 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:03.352603 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:03.354219 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:03.356012 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:03.356871 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:03.357025 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:03.362277 (+ 5252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:03.362287 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:03.362307 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:03.362361 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:03.365971 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:03.366831 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:03.366839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:03.366848 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:03.367055 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:03.367500 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:03.367504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:03.367506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:03.368155 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:03.368503 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:03.369002 (+ 499us) server_negotiation.cc:299] Negotiation successful
0928 16:37:03.369531 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":349,"threads_started":1}
I20260928 16:37:03.370819 14905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967: No bootstrap required, opened a new log
I20260928 16:37:03.380234 14905 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5561c411e6540e5b09298ca5dcae967" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 34961 } }
I20260928 16:37:03.380785 14905 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:03.380908 14905 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5561c411e6540e5b09298ca5dcae967, State: Initialized, Role: FOLLOWER
I20260928 16:37:03.381568 14905 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5561c411e6540e5b09298ca5dcae967" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 34961 } }
I20260928 16:37:03.381839 14905 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:03.381974 14905 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:03.382157 14905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:03.384671 14905 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5561c411e6540e5b09298ca5dcae967" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 34961 } }
I20260928 16:37:03.385295 14905 leader_election.cc:302] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5561c411e6540e5b09298ca5dcae967; no voters:
I20260928 16:37:03.385943 14905 leader_election.cc:288] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:03.386143 14910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:03.386809 14910 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [term 1 LEADER]: Becoming Leader. State: Replica: e5561c411e6540e5b09298ca5dcae967, State: Running, Role: LEADER
I20260928 16:37:03.387414 14910 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5561c411e6540e5b09298ca5dcae967" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 34961 } }
I20260928 16:37:03.388365 14905 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:03.390837 14911 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5561c411e6540e5b09298ca5dcae967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5561c411e6540e5b09298ca5dcae967" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 34961 } } }
I20260928 16:37:03.391084 14912 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5561c411e6540e5b09298ca5dcae967. Latest consensus state: current_term: 1 leader_uuid: "e5561c411e6540e5b09298ca5dcae967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5561c411e6540e5b09298ca5dcae967" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 34961 } } }
I20260928 16:37:03.391371 14911 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:03.391387 14912 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:03.394017 14919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:03.399892 14919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:03.409130 14919 catalog_manager.cc:1347] Generated new cluster ID: e199cd9043c640f8b778217976132fb1
I20260928 16:37:03.409253 14919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:03.428369 14919 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:03.430083 14919 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:03.442435 14919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5561c411e6540e5b09298ca5dcae967: Generated new TSK 0
I20260928 16:37:03.443537 14919 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:37:03.504424 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:34961
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:03.906749 14931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:03.907110 14931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:03.907215 14931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:03.915144 14931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:03.915289 14931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:03.915413 14931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:37:03.926380 14931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:34961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:03.928301 14931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:03.930629 14931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:03.950013 14936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:03.951021 14937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:03.952584 14939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:03.952598 14931 server_base.cc:1034] running on GCE node
I20260928 16:37:03.953505 14931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:03.954767 14931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:03.956036 14931 hybrid_clock.cc:648] HybridClock initialized: now 1790613423955972 us; error 51 us; skew 500 ppm
I20260928 16:37:03.956429 14931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:03 dist-test-slave-9fn6 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613423, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:03.960173 14931 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:37:03.961997 14931 webserver.cc:460] Webserver started at http://127.10.247.193:33435/ using document root <none> and password file <none>
I20260928 16:37:03.962853 14931 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:03.962987 14931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:03.963340 14931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:03.966876 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "0fdb95b26c0e401f93e0889021145b76"
format_stamp: "Formatted at 2026-09-28 16:37:03 on dist-test-slave-9fn6"
server_key: "10a2e57a27daae8df6e0e50f33cf5ffc"
server_key_iv: "f8bf2afd02ad33d7eb9e4fdba756f48d"
server_key_version: "encryptionkey@0"
I20260928 16:37:03.967824 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "0fdb95b26c0e401f93e0889021145b76"
format_stamp: "Formatted at 2026-09-28 16:37:03 on dist-test-slave-9fn6"
server_key: "10a2e57a27daae8df6e0e50f33cf5ffc"
server_key_iv: "f8bf2afd02ad33d7eb9e4fdba756f48d"
server_key_version: "encryptionkey@0"
I20260928 16:37:03.974499 14931 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260928 16:37:03.979013 14946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:03.980787 14931 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:03.980970 14931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "0fdb95b26c0e401f93e0889021145b76"
format_stamp: "Formatted at 2026-09-28 16:37:03 on dist-test-slave-9fn6"
server_key: "10a2e57a27daae8df6e0e50f33cf5ffc"
server_key_iv: "f8bf2afd02ad33d7eb9e4fdba756f48d"
server_key_version: "encryptionkey@0"
I20260928 16:37:03.981206 14931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:04.009536 14931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:04.015343 14931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:04.015709 14931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:04.016965 14931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:04.018620 14931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:04.018707 14931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:04.018854 14931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:04.018911 14931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:04.053428 14931 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:41455
I20260928 16:37:04.053473 15058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:41455 every 8 connection(s)
I20260928 16:37:04.054975 14931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:04.061935 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 14931
I20260928 16:37:04.062160 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:37:04.062825 11231 external_mini_cluster.cc:1428] Setting key 3a88cf500df084a7dccacf2519e575d6
Sep 28 16:37:04 dist-test-slave-9fn6 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613423, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:04.077215 15061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:04.057940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:40203 (local address 127.10.247.254:34961)
0928 16:37:04.058444 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:04.058450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:04.059507 (+ 1057us) server_negotiation.cc:407] Connection header received
0928 16:37:04.061676 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:04.061683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:04.061777 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:04.061884 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:04.063707 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:04.064545 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:04.066229 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:04.066662 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:04.066777 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:04.070764 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:04.070770 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:04.070774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:04.070815 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:04.073671 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:04.074512 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:04.074518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:04.074521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:04.074668 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:04.075124 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:04.075128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:04.075130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:04.075455 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:04.075608 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:04.076462 (+ 854us) server_negotiation.cc:299] Negotiation successful
0928 16:37:04.076783 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":249,"threads_started":1}
I20260928 16:37:04.078491 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:04.058480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:34961 (local address 127.10.247.193:40203)
0928 16:37:04.059316 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:04.059365 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:04.061339 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:04.062173 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:04.062183 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:04.062802 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:04.063510 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:04.063540 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:04.064730 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:04.064734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:04.066076 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:04.066090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:04.066892 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:04.066899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:04.067148 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:04.067915 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:04.067948 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:04.070497 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:04.073831 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:04.073842 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:04.073865 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:04.074379 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:04.074779 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:04.074785 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:04.074789 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:04.075005 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:04.075605 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:04.075615 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:04.076050 (+ 435us) client_negotiation.cc:769] Sending connection context
0928 16:37:04.076340 (+ 290us) client_negotiation.cc:240] Negotiation successful
0928 16:37:04.077091 (+ 751us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":303,"threads_started":1}
I20260928 16:37:04.080359 15059 heartbeater.cc:347] Connected to a master server at 127.10.247.254:34961
I20260928 16:37:04.080830 15059 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:04.081969 15059 heartbeater.cc:511] Master 127.10.247.254:34961 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
I20260928 16:37:04.084755 14870 ts_manager.cc:194] Registered new tserver with Master: 0fdb95b26c0e401f93e0889021145b76 (127.10.247.193:41455)
I20260928 16:37:04.086688 14870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:40203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:37:04.116460 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:34961
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:04.493597 15065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:04.493949 15065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:04.494064 15065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:04.501986 15065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:04.502131 15065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:04.502269 15065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:04.512899 15065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:34961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:04.514649 15065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:04.516297 15065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:04.532298 15070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:04.532480 15073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:04.532325 15071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:04.533169 15065 server_base.cc:1034] running on GCE node
I20260928 16:37:04.533890 15065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:04.535126 15065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:04.536353 15065 hybrid_clock.cc:648] HybridClock initialized: now 1790613424536293 us; error 46 us; skew 500 ppm
I20260928 16:37:04.536780 15065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:04 dist-test-slave-9fn6 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613424, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:04.540426 15065 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:37:04.542171 15065 webserver.cc:460] Webserver started at http://127.10.247.194:46609/ using document root <none> and password file <none>
I20260928 16:37:04.543013 15065 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:04.543130 15065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:04.543499 15065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:04.547083 15065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "43e55d360d714a859b3fb4dd9ac7f721"
format_stamp: "Formatted at 2026-09-28 16:37:04 on dist-test-slave-9fn6"
server_key: "ac48d64690c3eac2ec6ce7a76712cfa2"
server_key_iv: "7ae80a8f1ea882cd5774be0db014ff94"
server_key_version: "encryptionkey@0"
I20260928 16:37:04.547996 15065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "43e55d360d714a859b3fb4dd9ac7f721"
format_stamp: "Formatted at 2026-09-28 16:37:04 on dist-test-slave-9fn6"
server_key: "ac48d64690c3eac2ec6ce7a76712cfa2"
server_key_iv: "7ae80a8f1ea882cd5774be0db014ff94"
server_key_version: "encryptionkey@0"
I20260928 16:37:04.554262 15065 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260928 16:37:04.558225 15080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:04.559770 15065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:37:04.559943 15065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "43e55d360d714a859b3fb4dd9ac7f721"
format_stamp: "Formatted at 2026-09-28 16:37:04 on dist-test-slave-9fn6"
server_key: "ac48d64690c3eac2ec6ce7a76712cfa2"
server_key_iv: "7ae80a8f1ea882cd5774be0db014ff94"
server_key_version: "encryptionkey@0"
I20260928 16:37:04.560207 15065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:04.581187 15065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:04.585922 15065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:04.586228 15065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:04.587338 15065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:04.588928 15065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:04.589008 15065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:04.589136 15065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:04.589196 15065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:04.622586 15065 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:42667
I20260928 16:37:04.622619 15192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:42667 every 8 connection(s)
I20260928 16:37:04.624150 15065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:04.631774 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 15065
I20260928 16:37:04.632036 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:37:04.632517 11231 external_mini_cluster.cc:1428] Setting key 8662fc6cbae9c0e8c646cd8d4d38e588
Sep 28 16:37:04 dist-test-slave-9fn6 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613424, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:04.647040 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:04.627196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:50177 (local address 127.10.247.254:34961)
0928 16:37:04.627761 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:04.627768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:04.628990 (+ 1222us) server_negotiation.cc:407] Connection header received
0928 16:37:04.631070 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:04.631076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:04.631169 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:04.631278 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:04.633219 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:04.634170 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:04.635941 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:04.636418 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:04.636548 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:04.640477 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:04.640484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:04.640489 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:04.640548 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:04.643468 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:04.644340 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:04.644345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:04.644347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:04.644498 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:04.645009 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:04.645013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:04.645015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:04.645341 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:04.645496 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:04.646319 (+ 823us) server_negotiation.cc:299] Negotiation successful
0928 16:37:04.646623 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":264,"threads_started":1}
I20260928 16:37:04.648300 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:04.627897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:34961 (local address 127.10.247.194:50177)
0928 16:37:04.628796 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:04.628842 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:04.630733 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:04.631533 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:04.631544 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:04.632250 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:04.633019 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:04.633040 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:04.634334 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:04.634338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:04.635790 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:04.635803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:04.636712 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:04.636720 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:04.636974 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:04.637718 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:04.637752 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:04.640206 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:04.643643 (+ 3437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:04.643652 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:04.643670 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:04.644183 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:04.644656 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:04.644661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:04.644665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:04.644889 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:04.645469 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:04.645478 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:04.645902 (+ 424us) client_negotiation.cc:769] Sending connection context
0928 16:37:04.646215 (+ 313us) client_negotiation.cc:240] Negotiation successful
0928 16:37:04.646871 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":291,"threads_started":1}
I20260928 16:37:04.650250 15193 heartbeater.cc:347] Connected to a master server at 127.10.247.254:34961
I20260928 16:37:04.650637 15193 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:04.651543 15193 heartbeater.cc:511] Master 127.10.247.254:34961 requested a full tablet report, sending...
I20260928 16:37:04.653435 14870 ts_manager.cc:194] Registered new tserver with Master: 43e55d360d714a859b3fb4dd9ac7f721 (127.10.247.194:42667)
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
I20260928 16:37:04.654775 14870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:50177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:37:04.683513 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:34961
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:05.059434 15199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:05.059723 15199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:05.059864 15199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:05.067776 15199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:05.067921 15199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:05.068096 15199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:05.079205 15199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:34961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:05.081017 15199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:05.082638 15199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 16:37:05.090303 15059 heartbeater.cc:503] Master 127.10.247.254:34961 was elected leader, sending a full tablet report...
W20260928 16:37:05.098098 15204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:05.098121 15207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:05.098312 15205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:05.099076 15199 server_base.cc:1034] running on GCE node
I20260928 16:37:05.099787 15199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:05.100960 15199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:05.102212 15199 hybrid_clock.cc:648] HybridClock initialized: now 1790613425102147 us; error 48 us; skew 500 ppm
I20260928 16:37:05.102602 15199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:05 dist-test-slave-9fn6 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613425, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:05.106220 15199 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:37:05.107968 15199 webserver.cc:460] Webserver started at http://127.10.247.195:41179/ using document root <none> and password file <none>
I20260928 16:37:05.108865 15199 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:05.109007 15199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:05.109352 15199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:05.112835 15199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "738dbc5f666940e185dc1502c7a06046"
format_stamp: "Formatted at 2026-09-28 16:37:05 on dist-test-slave-9fn6"
server_key: "b5450aca1b01254f329cc3917cdf5eca"
server_key_iv: "9e33ae3c4eff3b883731dc1df2533891"
server_key_version: "encryptionkey@0"
I20260928 16:37:05.113739 15199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "738dbc5f666940e185dc1502c7a06046"
format_stamp: "Formatted at 2026-09-28 16:37:05 on dist-test-slave-9fn6"
server_key: "b5450aca1b01254f329cc3917cdf5eca"
server_key_iv: "9e33ae3c4eff3b883731dc1df2533891"
server_key_version: "encryptionkey@0"
I20260928 16:37:05.119892 15199 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260928 16:37:05.124436 15214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:05.126268 15199 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:05.126466 15199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "738dbc5f666940e185dc1502c7a06046"
format_stamp: "Formatted at 2026-09-28 16:37:05 on dist-test-slave-9fn6"
server_key: "b5450aca1b01254f329cc3917cdf5eca"
server_key_iv: "9e33ae3c4eff3b883731dc1df2533891"
server_key_version: "encryptionkey@0"
I20260928 16:37:05.126780 15199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:05.151643 15199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:05.157130 15199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:05.157441 15199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:05.158623 15199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:05.160260 15199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:05.160348 15199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:05.160445 15199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:05.160490 15199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:05.195838 15199 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:34569
I20260928 16:37:05.195883 15326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:34569 every 8 connection(s)
I20260928 16:37:05.197516 15199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:05.198493 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 15199
I20260928 16:37:05.198738 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:37:05.199448 11231 external_mini_cluster.cc:1428] Setting key 9f6f20e0312b0f6518b6e9bb56f574e0
Sep 28 16:37:05 dist-test-slave-9fn6 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613425, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:05.221141 15329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:05.201159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:56731 (local address 127.10.247.254:34961)
0928 16:37:05.201677 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:05.201684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:05.203183 (+ 1499us) server_negotiation.cc:407] Connection header received
0928 16:37:05.205483 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:05.205488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:05.205577 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:05.205684 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:05.207666 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.208581 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.210358 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.210791 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.210890 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:05.214921 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:05.214928 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:05.214933 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:05.214976 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:05.218010 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.218791 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.218795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.218798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.218931 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.219327 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.219330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.219332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.219634 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:05.219758 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:05.220516 (+ 758us) server_negotiation.cc:299] Negotiation successful
0928 16:37:05.220755 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":260,"threads_started":1}
I20260928 16:37:05.222322 15330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:05.202048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:34961 (local address 127.10.247.195:56731)
0928 16:37:05.202950 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:05.203015 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:05.205152 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:05.206025 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:05.206036 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:05.206720 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:05.207462 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:05.207485 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.208768 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.208772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:05.210220 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:05.210235 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.210991 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.210996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:05.211254 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:05.211956 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:05.211983 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:05.214650 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:05.218158 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:05.218167 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:05.218185 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:05.218694 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:05.219008 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:05.219011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:05.219014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:05.219246 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:05.219727 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:05.219734 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:05.220173 (+ 439us) client_negotiation.cc:769] Sending connection context
0928 16:37:05.220435 (+ 262us) client_negotiation.cc:240] Negotiation successful
0928 16:37:05.220969 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":325,"threads_started":1}
I20260928 16:37:05.223851 15327 heartbeater.cc:347] Connected to a master server at 127.10.247.254:34961
I20260928 16:37:05.224195 15327 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:05.225052 15327 heartbeater.cc:511] Master 127.10.247.254:34961 requested a full tablet report, sending...
I20260928 16:37:05.226688 14870 ts_manager.cc:194] Registered new tserver with Master: 738dbc5f666940e185dc1502c7a06046 (127.10.247.195:34569)
I20260928 16:37:05.227783 14870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:56731
I20260928 16:37:05.229733 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 16:37:05 dist-test-slave-9fn6 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 16:37:05 dist-test-slave-9fn6 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:37:05.290861 15335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:05.272638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.10.247.193:41455)
0928 16:37:05.273442 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:05.273451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:05.273485 (+ 34us) server_negotiation.cc:407] Connection header received
0928 16:37:05.273565 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:05.273573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:05.273853 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:05.274023 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:05.275719 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.277135 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.278564 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.279290 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.279405 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:05.283973 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:05.283992 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:05.284012 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:05.284066 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:05.287474 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.288342 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.288351 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.288354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.288526 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.289023 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.289027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.289030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.289512 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:05.289798 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:05.290223 (+ 425us) server_negotiation.cc:299] Negotiation successful
0928 16:37:05.290517 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":389,"threads_started":1}
Sep 28 16:37:05 dist-test-slave-9fn6 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:05.312005 15329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:05.296462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51524 (local address 127.10.247.254:34961)
0928 16:37:05.296784 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:05.296789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:05.296867 (+ 78us) server_negotiation.cc:407] Connection header received
0928 16:37:05.297081 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:05.297086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:05.297190 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:05.297283 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:05.298786 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.299722 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.301225 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.301713 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.301826 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:05.306310 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:05.306317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:05.306321 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:05.306371 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:05.309262 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.310060 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.310064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.310067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.310242 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.310584 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.310587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.310590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.310962 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:05.311209 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:05.311465 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 16:37:05.311676 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
W20260928 16:37:05.312896 14870 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51524
I20260928 16:37:05.314241 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14931
I20260928 16:37:05.328749 15335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:05.315349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.10.247.193:41455)
0928 16:37:05.315636 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:05.315642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:05.315781 (+ 139us) server_negotiation.cc:407] Connection header received
0928 16:37:05.315912 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:05.315916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:05.316040 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:05.316135 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:05.317541 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.318340 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.319795 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.320264 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.320370 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:05.323098 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:05.323103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:05.323107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:05.323151 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:05.325997 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.326748 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.326755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.326758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.326944 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.327324 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.327328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.327330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.327654 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:05.327881 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:05.328137 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 16:37:05.328361 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260928 16:37:05.329593 15054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:05.418715 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 14931
I20260928 16:37:05.435195 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 15065
Sep 28 16:37:05 dist-test-slave-9fn6 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:37:05.453485 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:05.436364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47250 (local address 127.10.247.194:42667)
0928 16:37:05.437019 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:05.437028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:05.437059 (+ 31us) server_negotiation.cc:407] Connection header received
0928 16:37:05.437140 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:05.437148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:05.437422 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:05.437604 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:05.439036 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.440188 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.441791 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.442462 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.442586 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:05.447139 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:05.447154 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:05.447174 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:05.447228 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:05.450425 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.451143 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.451151 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.451154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.451301 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.451722 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.451724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.451727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.452134 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:05.452446 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:05.452813 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 16:37:05.453077 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":332,"threads_started":1}
I20260928 16:37:05.455124 15188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:05.546142 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 15065
I20260928 16:37:05.562520 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 15199
Sep 28 16:37:05 dist-test-slave-9fn6 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:37:05.581409 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:05.563785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58500 (local address 127.10.247.195:34569)
0928 16:37:05.564502 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:05.564510 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:05.564547 (+ 37us) server_negotiation.cc:407] Connection header received
0928 16:37:05.564654 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:05.564664 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:05.564961 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:05.565113 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:05.566508 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.567649 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.569129 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:05.569774 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:05.569876 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:05.575024 (+ 5148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:05.575042 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:05.575061 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:05.575119 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:05.578501 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.579306 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.579315 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.579318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.579461 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:05.579799 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:05.579802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:05.579804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:05.580225 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:05.580503 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:05.580830 (+ 327us) server_negotiation.cc:299] Negotiation successful
0928 16:37:05.581084 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":347,"threads_started":1}
I20260928 16:37:05.583027 15322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:05.672230 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 15199
I20260928 16:37:05.689257 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 14838
I20260928 16:37:05.690824 14900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:05.830710 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 14838
2026-09-28T16:37:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4251 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:37:06 dist-test-slave-9fn6 krb5kdc[15347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:37:06 dist-test-slave-9fn6 krb5kdc[15347](info): set up 2 sockets
Sep 28 16:37:06 dist-test-slave-9fn6 krb5kdc[15347](info): commencing operation
krb5kdc: starting...
W20260928 16:37:07.489647 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.587s user 0.004s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:37:07 dist-test-slave-9fn6 krb5kdc[15347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:37:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:37:07.730649 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:46113
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:46113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:08.135025 15360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:08.135504 15360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:08.135617 15360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:08.144351 15360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:08.144546 15360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:08.144688 15360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:08.144785 15360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:08.144857 15360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:08.157503 15360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:46113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:46113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:08.159435 15360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:08.161386 15360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:08.176064 15365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:08.176167 15366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:08.176493 15368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:08.177052 15360 server_base.cc:1034] running on GCE node
I20260928 16:37:08.178270 15360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:08.180334 15360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:08.181711 15360 hybrid_clock.cc:648] HybridClock initialized: now 1790613428181652 us; error 69 us; skew 500 ppm
I20260928 16:37:08.182228 15360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:08 dist-test-slave-9fn6 krb5kdc[15347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:08.186333 15360 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:37:08.188000 15360 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:08.188136 15360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:08.188558 15360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:08.192723 15360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "826c59a567ea48b2a66d8a48c70d22d3"
format_stamp: "Formatted at 2026-09-28 16:37:08 on dist-test-slave-9fn6"
server_key: "716accf3ca879429ff6a0c8365df4c9a"
server_key_iv: "e9411686da1eb0cfe680a7ed208d4362"
server_key_version: "encryptionkey@0"
I20260928 16:37:08.193913 15360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "826c59a567ea48b2a66d8a48c70d22d3"
format_stamp: "Formatted at 2026-09-28 16:37:08 on dist-test-slave-9fn6"
server_key: "716accf3ca879429ff6a0c8365df4c9a"
server_key_iv: "e9411686da1eb0cfe680a7ed208d4362"
server_key_version: "encryptionkey@0"
I20260928 16:37:08.201647 15360 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260928 16:37:08.206758 15374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:08.208699 15360 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.007s sys 0.000s
I20260928 16:37:08.208936 15360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "826c59a567ea48b2a66d8a48c70d22d3"
format_stamp: "Formatted at 2026-09-28 16:37:08 on dist-test-slave-9fn6"
server_key: "716accf3ca879429ff6a0c8365df4c9a"
server_key_iv: "e9411686da1eb0cfe680a7ed208d4362"
server_key_version: "encryptionkey@0"
I20260928 16:37:08.209232 15360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:08.228196 15360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:08.233870 15360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:08.234201 15360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:08.255478 15360 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:46113
I20260928 16:37:08.255498 15425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:46113 every 8 connection(s)
I20260928 16:37:08.256705 15360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:08.258139 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 15360
I20260928 16:37:08.258411 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:08.259332 11231 external_mini_cluster.cc:1428] Setting key 5b40e6d9e0adbe03d54026a94ff566b0
I20260928 16:37:08.262590 15426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:08.277379 15426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3: Bootstrap starting.
Sep 28 16:37:08 dist-test-slave-9fn6 krb5kdc[15347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:08.282143 15426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:08.283420 15426 log.cc:824] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:08.287583 15426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3: No bootstrap required, opened a new log
I20260928 16:37:08.293131 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:08.262019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.10.247.254:46113)
0928 16:37:08.262955 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:08.262978 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:08.263031 (+ 53us) server_negotiation.cc:407] Connection header received
0928 16:37:08.265012 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:08.265041 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:08.265569 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:08.266021 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:08.268373 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:08.270089 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:08.272004 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:08.272869 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:08.273084 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:08.283191 (+ 10107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:08.283203 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:08.283226 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:08.283283 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:08.287274 (+ 3991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:08.288247 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:08.288257 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:08.288267 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:08.288513 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:08.288962 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:08.288967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:08.288970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:08.289842 (+ 872us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:08.290203 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:08.290755 (+ 552us) server_negotiation.cc:299] Negotiation successful
0928 16:37:08.291364 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":431,"threads_started":1}
I20260928 16:37:08.297698 15426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826c59a567ea48b2a66d8a48c70d22d3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46113 } }
I20260928 16:37:08.298130 15426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:08.298211 15426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 826c59a567ea48b2a66d8a48c70d22d3, State: Initialized, Role: FOLLOWER
I20260928 16:37:08.298815 15426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826c59a567ea48b2a66d8a48c70d22d3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46113 } }
I20260928 16:37:08.299062 15426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:08.299201 15426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:08.299365 15426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:08.302006 15426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826c59a567ea48b2a66d8a48c70d22d3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46113 } }
I20260928 16:37:08.302695 15426 leader_election.cc:302] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 826c59a567ea48b2a66d8a48c70d22d3; no voters:
I20260928 16:37:08.303411 15426 leader_election.cc:288] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:08.303613 15431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:08.304421 15431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [term 1 LEADER]: Becoming Leader. State: Replica: 826c59a567ea48b2a66d8a48c70d22d3, State: Running, Role: LEADER
I20260928 16:37:08.305125 15431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826c59a567ea48b2a66d8a48c70d22d3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46113 } }
I20260928 16:37:08.305920 15426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:08.309496 15433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 826c59a567ea48b2a66d8a48c70d22d3. Latest consensus state: current_term: 1 leader_uuid: "826c59a567ea48b2a66d8a48c70d22d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826c59a567ea48b2a66d8a48c70d22d3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46113 } } }
I20260928 16:37:08.309950 15433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:08.310984 15437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:08.311527 15432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "826c59a567ea48b2a66d8a48c70d22d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826c59a567ea48b2a66d8a48c70d22d3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 46113 } } }
I20260928 16:37:08.311941 15432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:08.318907 15437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:08.331535 15437 catalog_manager.cc:1347] Generated new cluster ID: 30e52472c43549b9afe8b189b92255ce
I20260928 16:37:08.331683 15437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:08.353853 15437 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:08.355609 15437 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:08.371132 15437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3: Generated new TSK 0
I20260928 16:37:08.372588 15437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:37:08.437594 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:46113
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:08.839277 15452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:08.839689 15452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:08.839836 15452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:08.847892 15452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:08.848117 15452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:08.848305 15452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:37:08.859724 15452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:46113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:08.861593 15452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:08.863389 15452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:08.878471 15457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:08.878504 15458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:08.878521 15460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:08.879244 15452 server_base.cc:1034] running on GCE node
I20260928 16:37:08.879966 15452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:08.881310 15452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:08.882529 15452 hybrid_clock.cc:648] HybridClock initialized: now 1790613428882470 us; error 66 us; skew 500 ppm
I20260928 16:37:08.882907 15452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:08 dist-test-slave-9fn6 krb5kdc[15347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:08.886662 15452 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:37:08.888136 15452 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:08.888263 15452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:08.888692 15452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:08.892452 15452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6"
format_stamp: "Formatted at 2026-09-28 16:37:08 on dist-test-slave-9fn6"
server_key: "0c46c670c42cffa835b0802331d247a3"
server_key_iv: "91af01aaa7779d1716851ac758d90b4e"
server_key_version: "encryptionkey@0"
I20260928 16:37:08.893532 15452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6"
format_stamp: "Formatted at 2026-09-28 16:37:08 on dist-test-slave-9fn6"
server_key: "0c46c670c42cffa835b0802331d247a3"
server_key_iv: "91af01aaa7779d1716851ac758d90b4e"
server_key_version: "encryptionkey@0"
I20260928 16:37:08.900149 15452 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:37:08.904474 15466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:08.906270 15452 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:08.906471 15452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6"
format_stamp: "Formatted at 2026-09-28 16:37:08 on dist-test-slave-9fn6"
server_key: "0c46c670c42cffa835b0802331d247a3"
server_key_iv: "91af01aaa7779d1716851ac758d90b4e"
server_key_version: "encryptionkey@0"
I20260928 16:37:08.906755 15452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:08.930744 15452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:08.935667 15452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:08.935977 15452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:08.937196 15452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:08.938828 15452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:08.938941 15452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:08.939055 15452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:08.939128 15452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:08.974908 15452 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:35825
I20260928 16:37:08.974954 15578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:35825 every 8 connection(s)
I20260928 16:37:08.976025 15452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:08.977113 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 15452
I20260928 16:37:08.977389 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:37:08.978322 11231 external_mini_cluster.cc:1428] Setting key 266cec5aee06d5821f9aaa091bf86d89
Sep 28 16:37:08 dist-test-slave-9fn6 krb5kdc[15347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613428, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:09.000813 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:08.979509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:37571 (local address 127.10.247.254:46113)
0928 16:37:08.980724 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:08.980731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:08.981562 (+ 831us) server_negotiation.cc:407] Connection header received
0928 16:37:08.983717 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:08.983723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:08.983816 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:08.983943 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:08.985799 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:08.986851 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:08.989036 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:08.989498 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:08.989614 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:08.994139 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:08.994145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:08.994149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:08.994192 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:08.997041 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:08.998019 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:08.998023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:08.998026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:08.998176 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:08.998659 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:08.998663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:08.998665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:08.999000 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:08.999166 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:09.000050 (+ 884us) server_negotiation.cc:299] Negotiation successful
0928 16:37:09.000349 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":907,"threads_started":1}
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
I20260928 16:37:09.002310 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:08.980378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46113 (local address 127.10.247.193:37571)
0928 16:37:08.981358 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:08.981418 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:08.983389 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:08.984170 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:08.984181 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:08.984878 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:08.985611 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:08.985632 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:08.987045 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:08.987050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:08.988871 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:08.988889 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:08.989758 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:08.989764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:08.990057 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:08.990868 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:08.990901 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:08.993836 (+ 2935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:08.997234 (+ 3398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:08.997242 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:08.997262 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:08.997862 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:08.998293 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:08.998298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:08.998302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:08.998533 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:08.999132 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:08.999140 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:08.999597 (+ 457us) client_negotiation.cc:769] Sending connection context
0928 16:37:08.999885 (+ 288us) client_negotiation.cc:240] Negotiation successful
0928 16:37:09.000651 (+ 766us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":402,"threads_started":1}
I20260928 16:37:09.004238 15579 heartbeater.cc:347] Connected to a master server at 127.10.247.254:46113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
I20260928 16:37:09.004635 15579 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:09.005625 15579 heartbeater.cc:511] Master 127.10.247.254:46113 requested a full tablet report, sending...
I20260928 16:37:09.008643 15391 ts_manager.cc:194] Registered new tserver with Master: 2d3741488c2a4dbdb5e7fb3f6efea3d6 (127.10.247.193:35825)
I20260928 16:37:09.010366 15391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:37571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:37:09.033536 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:46113
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:09.417863 15585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:09.418224 15585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:09.418362 15585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:09.426385 15585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:09.426540 15585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:09.426713 15585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:09.438290 15585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:46113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:09.440083 15585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:09.441812 15585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:09.456712 15591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:09.456753 15590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:09.456784 15593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:09.457525 15585 server_base.cc:1034] running on GCE node
I20260928 16:37:09.458283 15585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:09.459457 15585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:09.460734 15585 hybrid_clock.cc:648] HybridClock initialized: now 1790613429460664 us; error 56 us; skew 500 ppm
I20260928 16:37:09.461166 15585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:09 dist-test-slave-9fn6 krb5kdc[15347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613429, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:09.464931 15585 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:37:09.466399 15585 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:09.466516 15585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:09.466877 15585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:09.470628 15585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "e8c40d2b88c148e88e695aa80624316b"
format_stamp: "Formatted at 2026-09-28 16:37:09 on dist-test-slave-9fn6"
server_key: "4205ef89a5e879bcfd3dcd6fc41fac5c"
server_key_iv: "1a3a7a3b0f6a42fce63bed0ac23fe440"
server_key_version: "encryptionkey@0"
I20260928 16:37:09.471609 15585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "e8c40d2b88c148e88e695aa80624316b"
format_stamp: "Formatted at 2026-09-28 16:37:09 on dist-test-slave-9fn6"
server_key: "4205ef89a5e879bcfd3dcd6fc41fac5c"
server_key_iv: "1a3a7a3b0f6a42fce63bed0ac23fe440"
server_key_version: "encryptionkey@0"
I20260928 16:37:09.478188 15585 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260928 16:37:09.482472 15599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:09.484126 15585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:37:09.484320 15585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "e8c40d2b88c148e88e695aa80624316b"
format_stamp: "Formatted at 2026-09-28 16:37:09 on dist-test-slave-9fn6"
server_key: "4205ef89a5e879bcfd3dcd6fc41fac5c"
server_key_iv: "1a3a7a3b0f6a42fce63bed0ac23fe440"
server_key_version: "encryptionkey@0"
I20260928 16:37:09.484562 15585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:09.512830 15585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:09.518354 15585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:09.518743 15585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:09.520067 15585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:09.521950 15585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:09.522080 15585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:09.522218 15585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:09.522293 15585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:09.558097 15585 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:43391
I20260928 16:37:09.558148 15712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:43391 every 8 connection(s)
I20260928 16:37:09.559182 15585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:09.559819 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 15585
I20260928 16:37:09.560060 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:37:09.560760 11231 external_mini_cluster.cc:1428] Setting key 682fc5a38fc25396d717e745ee358676
Sep 28 16:37:09 dist-test-slave-9fn6 krb5kdc[15347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613429, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:09.585899 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:09.562237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:54497 (local address 127.10.247.254:46113)
0928 16:37:09.562739 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:09.562744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:09.564355 (+ 1611us) server_negotiation.cc:407] Connection header received
0928 16:37:09.567202 (+ 2847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:09.567207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:09.567293 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:09.567399 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:09.569951 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:09.570814 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:09.573096 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:09.573804 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:09.573956 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:09.578568 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:09.578574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:09.578577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:09.578623 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:09.581566 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:09.582789 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:09.582794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:09.582797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:09.582957 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:09.583608 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:09.583612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:09.583614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:09.583976 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:09.584130 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:09.585353 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0928 16:37:09.585603 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":269,"threads_started":1}
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
I20260928 16:37:09.587637 15716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:09.563135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46113 (local address 127.10.247.194:54497)
0928 16:37:09.564131 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:09.564214 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:09.566826 (+ 2612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:09.567710 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:09.567725 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:09.568596 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:09.569697 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:09.569726 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:09.571016 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:09.571023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:09.572936 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:09.572951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:09.574101 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:09.574107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:09.574338 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:09.575168 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:09.575197 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:09.578223 (+ 3026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:09.581763 (+ 3540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:09.581776 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:09.581805 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:09.582630 (+ 825us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:09.583098 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:09.583105 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:09.583109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:09.583422 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:09.584124 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:09.584135 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:09.584816 (+ 681us) client_negotiation.cc:769] Sending connection context
0928 16:37:09.585193 (+ 377us) client_negotiation.cc:240] Negotiation successful
0928 16:37:09.585933 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":369,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
I20260928 16:37:09.589669 15713 heartbeater.cc:347] Connected to a master server at 127.10.247.254:46113
I20260928 16:37:09.590065 15713 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:09.591071 15713 heartbeater.cc:511] Master 127.10.247.254:46113 requested a full tablet report, sending...
I20260928 16:37:09.593170 15391 ts_manager.cc:194] Registered new tserver with Master: e8c40d2b88c148e88e695aa80624316b (127.10.247.194:43391)
I20260928 16:37:09.594156 15391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:54497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:37:09.620868 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:46113
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:09.998974 15719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:09.999358 15719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:09.999500 15719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:10.007714 15719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:10.007867 15719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:10.008056 15719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:10.014654 15579 heartbeater.cc:503] Master 127.10.247.254:46113 was elected leader, sending a full tablet report...
I20260928 16:37:10.020293 15719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:46113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:10.022217 15719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:10.024027 15719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:10.040136 15725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:10.040230 15724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:10.040249 15727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:10.040835 15719 server_base.cc:1034] running on GCE node
I20260928 16:37:10.041791 15719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:10.042963 15719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:10.044170 15719 hybrid_clock.cc:648] HybridClock initialized: now 1790613430044119 us; error 43 us; skew 500 ppm
I20260928 16:37:10.044544 15719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:10 dist-test-slave-9fn6 krb5kdc[15347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613430, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:10.048064 15719 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:37:10.049449 15719 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:10.049557 15719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:10.049875 15719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:10.053541 15719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "2cc6eb21cb5b4cf29e99253daeda9d37"
format_stamp: "Formatted at 2026-09-28 16:37:10 on dist-test-slave-9fn6"
server_key: "81dc879e835c5f84ae423d7fddb65284"
server_key_iv: "0418e5e3aa79dcc19c5f83691dd8356f"
server_key_version: "encryptionkey@0"
I20260928 16:37:10.054530 15719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "2cc6eb21cb5b4cf29e99253daeda9d37"
format_stamp: "Formatted at 2026-09-28 16:37:10 on dist-test-slave-9fn6"
server_key: "81dc879e835c5f84ae423d7fddb65284"
server_key_iv: "0418e5e3aa79dcc19c5f83691dd8356f"
server_key_version: "encryptionkey@0"
I20260928 16:37:10.060724 15719 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:37:10.064919 15733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:10.066694 15719 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:37:10.066867 15719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "2cc6eb21cb5b4cf29e99253daeda9d37"
format_stamp: "Formatted at 2026-09-28 16:37:10 on dist-test-slave-9fn6"
server_key: "81dc879e835c5f84ae423d7fddb65284"
server_key_iv: "0418e5e3aa79dcc19c5f83691dd8356f"
server_key_version: "encryptionkey@0"
I20260928 16:37:10.067106 15719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:10.092383 15719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:10.097838 15719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:10.098169 15719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:10.099376 15719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:10.101200 15719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:10.101302 15719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:10.101410 15719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:10.101495 15719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:10.136901 15719 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:43745
I20260928 16:37:10.136987 15845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:43745 every 8 connection(s)
I20260928 16:37:10.137935 15719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:10.146622 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 15719
I20260928 16:37:10.146850 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:37:10.147352 11231 external_mini_cluster.cc:1428] Setting key abf6adb4a97675ae84681755f79c78ae
Sep 28 16:37:10 dist-test-slave-9fn6 krb5kdc[15347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613430, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:10.161898 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.140822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:48565 (local address 127.10.247.254:46113)
0928 16:37:10.141351 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.141357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.142526 (+ 1169us) server_negotiation.cc:407] Connection header received
0928 16:37:10.144732 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.144740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.144862 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.145002 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:10.147015 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.147963 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.150029 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.150656 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.150785 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.155074 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:10.155080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:10.155084 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:10.155129 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:10.158086 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.158952 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.158957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.158960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.159108 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.159558 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.159561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.159563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.159904 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:10.160080 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.161196 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.161465 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":247,"threads_started":1}
I20260928 16:37:10.163115 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.141408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:46113 (local address 127.10.247.195:48565)
0928 16:37:10.142283 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:10.142338 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:10.144356 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:10.145292 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:10.145302 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:10.145992 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:10.146764 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.146795 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.148150 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.148155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.149879 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.149898 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.151115 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.151121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.151355 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.152090 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:10.152117 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:10.154772 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:10.158256 (+ 3484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:10.158268 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:10.158289 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:10.158804 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:10.159217 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:10.159222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:10.159226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:10.159435 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:10.160054 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:10.160064 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:10.160659 (+ 595us) client_negotiation.cc:769] Sending connection context
0928 16:37:10.161051 (+ 392us) client_negotiation.cc:240] Negotiation successful
0928 16:37:10.161674 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":333,"threads_started":1}
I20260928 16:37:10.164865 15846 heartbeater.cc:347] Connected to a master server at 127.10.247.254:46113
I20260928 16:37:10.165244 15846 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:10.166060 15846 heartbeater.cc:511] Master 127.10.247.254:46113 requested a full tablet report, sending...
I20260928 16:37:10.167888 15391 ts_manager.cc:194] Registered new tserver with Master: 2cc6eb21cb5b4cf29e99253daeda9d37 (127.10.247.195:43745)
I20260928 16:37:10.168905 15391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:48565
I20260928 16:37:10.176962 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:37:10.197393 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.182650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.10.247.254:46113)
0928 16:37:10.182908 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.182917 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.183401 (+ 484us) server_negotiation.cc:407] Connection header received
0928 16:37:10.183590 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.183595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.183689 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.183780 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:10.185428 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.186366 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.187730 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.188274 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.188378 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.191390 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:10.191396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:10.191400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:10.191448 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:10.194339 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.195111 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.195116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.195118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.195268 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.195679 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.195683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.195685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.196112 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:10.196336 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.196803 (+ 467us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.197066 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:37:10.202766 15391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:37:10.211298 15391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:37:10.240338 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.226972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:43391 (local address 127.0.0.1:39930)
0928 16:37:10.228686 (+ 1714us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:10.228754 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:10.228989 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:10.231468 (+ 2479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:10.231476 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:10.231519 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:10.231997 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.232012 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.234407 (+ 2395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.234413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.237067 (+ 2654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.237083 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.239175 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.239181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.239408 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.239467 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 16:37:10.239601 (+ 134us) client_negotiation.cc:240] Negotiation successful
0928 16:37:10.239838 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1334,"thread_start_us":3611,"threads_started":1}
I20260928 16:37:10.241016 15862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.228151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.10.247.193:35825)
0928 16:37:10.231775 (+ 3624us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.231785 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.231816 (+ 31us) server_negotiation.cc:407] Connection header received
0928 16:37:10.231931 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.231940 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.232193 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.233136 (+ 943us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:10.233393 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.234892 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.237469 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.238931 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.239240 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.239481 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.240415 (+ 934us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.240647 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3396,"thread_start_us":3464,"threads_started":1}
I20260928 16:37:10.241463 15859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.227357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:35825 (local address 127.0.0.1:34208)
0928 16:37:10.230888 (+ 3531us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:10.230907 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:10.231031 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:10.232416 (+ 1385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:10.232421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:10.232455 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:10.232932 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.232947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.235046 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.235051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.237256 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.237269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.239599 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.239609 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.239785 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.239817 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 16:37:10.239885 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 16:37:10.240015 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3412,"thread_start_us":1074,"threads_started":1}
I20260928 16:37:10.242712 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.227248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.10.247.194:43391)
0928 16:37:10.229726 (+ 2478us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.229740 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.229779 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:37:10.229911 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.229922 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.231177 (+ 1255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.232212 (+ 1035us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:10.232541 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.234247 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.237308 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.238900 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.241858 (+ 2958us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.242096 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.242175 (+ 79us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.242376 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2258,"thread_start_us":2083,"threads_started":1}
I20260928 16:37:10.246933 15514 tablet_service.cc:1467] Processing CreateTablet for tablet 836bd460f34d48bdbfc7126ea9c66786 (DEFAULT_TABLE table=test-table [id=95ca06e8f4f74dfc94276f174f8a26f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:10.246942 15648 tablet_service.cc:1467] Processing CreateTablet for tablet 836bd460f34d48bdbfc7126ea9c66786 (DEFAULT_TABLE table=test-table [id=95ca06e8f4f74dfc94276f174f8a26f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:10.248948 15648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836bd460f34d48bdbfc7126ea9c66786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:10.248939 15514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836bd460f34d48bdbfc7126ea9c66786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:10.251199 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.230555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:43745 (local address 127.0.0.1:55946)
0928 16:37:10.240694 (+ 10139us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:10.240717 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:10.240838 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:10.245041 (+ 4203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:10.245046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:10.245078 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:10.245342 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.245359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.246917 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.246923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.248742 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.248756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.250254 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.250261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.250454 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.250491 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 16:37:10.250586 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 16:37:10.250767 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10011,"thread_start_us":12037,"threads_started":1}
I20260928 16:37:10.251223 15864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.231319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55946 (local address 127.10.247.195:43745)
0928 16:37:10.244341 (+ 13022us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.244350 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.244390 (+ 40us) server_negotiation.cc:407] Connection header received
0928 16:37:10.244513 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.244522 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.244815 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.245040 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:10.245537 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.246722 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.248914 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.250061 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.250266 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.250492 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.250660 (+ 168us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.250857 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12766,"thread_start_us":12871,"threads_started":1}
I20260928 16:37:10.256735 15781 tablet_service.cc:1467] Processing CreateTablet for tablet 836bd460f34d48bdbfc7126ea9c66786 (DEFAULT_TABLE table=test-table [id=95ca06e8f4f74dfc94276f174f8a26f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:10.258846 15781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836bd460f34d48bdbfc7126ea9c66786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:10.277565 15865 tablet_bootstrap.cc:492] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: Bootstrap starting.
I20260928 16:37:10.279260 15866 tablet_bootstrap.cc:492] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: Bootstrap starting.
I20260928 16:37:10.280226 15867 tablet_bootstrap.cc:492] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: Bootstrap starting.
I20260928 16:37:10.284536 15865 tablet_bootstrap.cc:654] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:10.284580 15866 tablet_bootstrap.cc:654] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:10.284593 15867 tablet_bootstrap.cc:654] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:10.285799 15865 log.cc:824] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:10.285913 15867 log.cc:824] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:10.285913 15866 log.cc:824] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:10.289557 15866 tablet_bootstrap.cc:492] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: No bootstrap required, opened a new log
I20260928 16:37:10.289543 15867 tablet_bootstrap.cc:492] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: No bootstrap required, opened a new log
I20260928 16:37:10.289542 15865 tablet_bootstrap.cc:492] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: No bootstrap required, opened a new log
I20260928 16:37:10.289875 15867 ts_tablet_manager.cc:1397] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260928 16:37:10.289881 15865 ts_tablet_manager.cc:1397] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.008s
I20260928 16:37:10.289937 15866 ts_tablet_manager.cc:1397] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260928 16:37:10.298130 15865 raft_consensus.cc:348] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } }
I20260928 16:37:10.298686 15865 raft_consensus.cc:374] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:10.298816 15865 raft_consensus.cc:729] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8c40d2b88c148e88e695aa80624316b, State: Initialized, Role: FOLLOWER
I20260928 16:37:10.299629 15865 consensus_queue.cc:260] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } }
I20260928 16:37:10.301592 15866 raft_consensus.cc:348] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } }
I20260928 16:37:10.301604 15867 raft_consensus.cc:348] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } }
I20260928 16:37:10.302143 15866 raft_consensus.cc:374] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:10.302160 15867 raft_consensus.cc:374] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:10.302255 15866 raft_consensus.cc:729] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d3741488c2a4dbdb5e7fb3f6efea3d6, State: Initialized, Role: FOLLOWER
I20260928 16:37:10.302255 15867 raft_consensus.cc:729] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc6eb21cb5b4cf29e99253daeda9d37, State: Initialized, Role: FOLLOWER
I20260928 16:37:10.303105 15866 consensus_queue.cc:260] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } }
I20260928 16:37:10.303560 15713 heartbeater.cc:503] Master 127.10.247.254:46113 was elected leader, sending a full tablet report...
I20260928 16:37:10.304505 15865 ts_tablet_manager.cc:1428] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260928 16:37:10.303104 15867 consensus_queue.cc:260] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } }
I20260928 16:37:10.305032 15866 ts_tablet_manager.cc:1428] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260928 16:37:10.306180 15846 heartbeater.cc:503] Master 127.10.247.254:46113 was elected leader, sending a full tablet report...
I20260928 16:37:10.306865 15867 ts_tablet_manager.cc:1428] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: Time spent starting tablet: real 0.017s user 0.013s sys 0.002s
W20260928 16:37:10.311570 15714 tablet.cc:2369] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:37:10.389788 15847 tablet.cc:2369] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:37:10.457724 15873 raft_consensus.cc:482] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:10.458127 15873 raft_consensus.cc:504] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } }
I20260928 16:37:10.460253 15873 leader_election.cc:288] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8c40d2b88c148e88e695aa80624316b (127.10.247.194:43391), 2d3741488c2a4dbdb5e7fb3f6efea3d6 (127.10.247.193:35825)
I20260928 16:37:10.468346 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.460841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:47539 (local address 127.10.247.194:43391)
0928 16:37:10.461125 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.461132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.461319 (+ 187us) server_negotiation.cc:407] Connection header received
0928 16:37:10.461577 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.461583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.461676 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.461780 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:10.462356 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.463451 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.465994 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.467281 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.467453 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.467543 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.467875 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.468033 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260928 16:37:10.468370 15862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.460917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:42975 (local address 127.10.247.193:35825)
0928 16:37:10.461287 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.461293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.461951 (+ 658us) server_negotiation.cc:407] Connection header received
0928 16:37:10.462114 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.462118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.462183 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.462254 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:10.462789 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.463785 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.466198 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.467440 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.467623 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.467681 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.467971 (+ 290us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.468093 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260928 16:37:10.468462 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.460866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:43391 (local address 127.10.247.195:47539)
0928 16:37:10.461202 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:10.461224 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:10.461389 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:10.461832 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:10.461838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:10.461881 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:10.462200 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.462211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.463601 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.463605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.465813 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.465828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.467451 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.467456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.467698 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.467740 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 16:37:10.467838 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 16:37:10.468059 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":36,"spinlock_wait_cycles":30464}
I20260928 16:37:10.468550 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.461035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:35825 (local address 127.10.247.195:42975)
0928 16:37:10.461836 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:10.461853 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:10.461990 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:10.462294 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:10.462299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:10.462324 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:10.462640 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.462649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.463918 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.463922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.466048 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:10.466059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.467634 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.467639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:10.467817 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.467846 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 16:37:10.467916 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 16:37:10.468047 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":80,"thread_start_us":466,"threads_started":1}
I20260928 16:37:10.469837 15668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836bd460f34d48bdbfc7126ea9c66786" candidate_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8c40d2b88c148e88e695aa80624316b" is_pre_election: true
I20260928 16:37:10.469838 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836bd460f34d48bdbfc7126ea9c66786" candidate_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" is_pre_election: true
I20260928 16:37:10.470453 15668 raft_consensus.cc:2454] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cc6eb21cb5b4cf29e99253daeda9d37 in term 0.
I20260928 16:37:10.470453 15534 raft_consensus.cc:2454] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cc6eb21cb5b4cf29e99253daeda9d37 in term 0.
I20260928 16:37:10.471200 15735 leader_election.cc:302] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc6eb21cb5b4cf29e99253daeda9d37, e8c40d2b88c148e88e695aa80624316b; no voters:
I20260928 16:37:10.471678 15873 raft_consensus.cc:2781] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:10.471891 15873 raft_consensus.cc:482] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:10.472012 15873 raft_consensus.cc:3037] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:10.474604 15873 raft_consensus.cc:504] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } }
I20260928 16:37:10.475337 15873 leader_election.cc:288] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [CANDIDATE]: Term 1 election: Requested vote from peers e8c40d2b88c148e88e695aa80624316b (127.10.247.194:43391), 2d3741488c2a4dbdb5e7fb3f6efea3d6 (127.10.247.193:35825)
I20260928 16:37:10.475891 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836bd460f34d48bdbfc7126ea9c66786" candidate_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6"
I20260928 16:37:10.475883 15668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836bd460f34d48bdbfc7126ea9c66786" candidate_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8c40d2b88c148e88e695aa80624316b"
I20260928 16:37:10.476135 15668 raft_consensus.cc:3037] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:10.476143 15534 raft_consensus.cc:3037] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:10.478911 15668 raft_consensus.cc:2454] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc6eb21cb5b4cf29e99253daeda9d37 in term 1.
W20260928 16:37:10.478916 15580 tablet.cc:2369] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:37:10.478888 15534 raft_consensus.cc:2454] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc6eb21cb5b4cf29e99253daeda9d37 in term 1.
I20260928 16:37:10.479373 15735 leader_election.cc:302] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc6eb21cb5b4cf29e99253daeda9d37, e8c40d2b88c148e88e695aa80624316b; no voters:
I20260928 16:37:10.479719 15873 raft_consensus.cc:2781] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:10.480274 15873 raft_consensus.cc:686] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 1 LEADER]: Becoming Leader. State: Replica: 2cc6eb21cb5b4cf29e99253daeda9d37, State: Running, Role: LEADER
I20260928 16:37:10.480847 15873 consensus_queue.cc:237] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } }
I20260928 16:37:10.486500 15390 catalog_manager.cc:5495] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cc6eb21cb5b4cf29e99253daeda9d37 (127.10.247.195). New cstate: current_term: 1 leader_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cc6eb21cb5b4cf29e99253daeda9d37" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43745 } health_report { overall_health: HEALTHY } } }
I20260928 16:37:10.505821 15864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.499973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.10.247.195:43745)
0928 16:37:10.500204 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.500209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.500293 (+ 84us) server_negotiation.cc:407] Connection header received
0928 16:37:10.500518 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.500522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.500681 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.500831 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:37:10.501205 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.502034 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.503429 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.503846 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.503941 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.504475 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:37:10.505235 (+ 760us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:37:10.505362 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.505434 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.505525 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260928 16:37:10.516394 15668 raft_consensus.cc:1261] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [term 1 FOLLOWER]: Refusing update from remote peer 2cc6eb21cb5b4cf29e99253daeda9d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:10.516394 15534 raft_consensus.cc:1261] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [term 1 FOLLOWER]: Refusing update from remote peer 2cc6eb21cb5b4cf29e99253daeda9d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:10.517585 15875 consensus_queue.cc:1035] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d3741488c2a4dbdb5e7fb3f6efea3d6" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:10.518013 15873 consensus_queue.cc:1035] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8c40d2b88c148e88e695aa80624316b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 43391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:10.549189 15390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:10.549614 15390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:10.554829 15390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 826c59a567ea48b2a66d8a48c70d22d3: Sending DeleteTablet for 3 replicas of tablet 836bd460f34d48bdbfc7126ea9c66786
I20260928 16:37:10.556444 15648 tablet_service.cc:1514] Processing DeleteTablet for tablet 836bd460f34d48bdbfc7126ea9c66786 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:10 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:39930
I20260928 16:37:10.556815 15514 tablet_service.cc:1514] Processing DeleteTablet for tablet 836bd460f34d48bdbfc7126ea9c66786 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:10 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:34208
I20260928 16:37:10.556453 15781 tablet_service.cc:1514] Processing DeleteTablet for tablet 836bd460f34d48bdbfc7126ea9c66786 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:10 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:55946
I20260928 16:37:10.570326 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 15452
I20260928 16:37:10.573439 15888 tablet_replica.cc:330] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: stopping tablet replica
I20260928 16:37:10.574122 15888 raft_consensus.cc:2229] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:10.574904 15888 raft_consensus.cc:2258] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:10.576551 15890 tablet_replica.cc:330] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: stopping tablet replica
I20260928 16:37:10.577172 15890 raft_consensus.cc:2229] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:37:10.578025 15890 raft_consensus.cc:2258] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:10.580502 15890 ts_tablet_manager.cc:1905] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:10.580514 15888 ts_tablet_manager.cc:1905] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:10.581272 15889 tablet_replica.cc:330] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: stopping tablet replica
I20260928 16:37:10.581861 15889 raft_consensus.cc:2229] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:10.582499 15889 raft_consensus.cc:2258] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:10.585040 15889 ts_tablet_manager.cc:1905] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:10.588193 15890 ts_tablet_manager.cc:1918] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:10.588388 15890 log.cc:1196] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/836bd460f34d48bdbfc7126ea9c66786
I20260928 16:37:10.588979 15890 ts_tablet_manager.cc:1939] T 836bd460f34d48bdbfc7126ea9c66786 P 2cc6eb21cb5b4cf29e99253daeda9d37: Deleting consensus metadata
I20260928 16:37:10.591149 15375 catalog_manager.cc:4841] TS 2cc6eb21cb5b4cf29e99253daeda9d37 (127.10.247.195:43745): tablet 836bd460f34d48bdbfc7126ea9c66786 (table test-table [id=95ca06e8f4f74dfc94276f174f8a26f7]) successfully deleted
I20260928 16:37:10.596817 15889 ts_tablet_manager.cc:1918] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:10.597054 15889 log.cc:1196] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/836bd460f34d48bdbfc7126ea9c66786
I20260928 16:37:10.597632 15889 ts_tablet_manager.cc:1939] T 836bd460f34d48bdbfc7126ea9c66786 P 2d3741488c2a4dbdb5e7fb3f6efea3d6: Deleting consensus metadata
I20260928 16:37:10.598563 15888 ts_tablet_manager.cc:1918] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:10.598840 15888 log.cc:1196] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/836bd460f34d48bdbfc7126ea9c66786
I20260928 16:37:10.599478 15888 ts_tablet_manager.cc:1939] T 836bd460f34d48bdbfc7126ea9c66786 P e8c40d2b88c148e88e695aa80624316b: Deleting consensus metadata
I20260928 16:37:10.600039 15378 catalog_manager.cc:4841] TS 2d3741488c2a4dbdb5e7fb3f6efea3d6 (127.10.247.193:35825): tablet 836bd460f34d48bdbfc7126ea9c66786 (table test-table [id=95ca06e8f4f74dfc94276f174f8a26f7]) successfully deleted
I20260928 16:37:10.602146 15376 catalog_manager.cc:4841] TS e8c40d2b88c148e88e695aa80624316b (127.10.247.194:43391): tablet 836bd460f34d48bdbfc7126ea9c66786 (table test-table [id=95ca06e8f4f74dfc94276f174f8a26f7]) successfully deleted
Sep 28 16:37:10 dist-test-slave-9fn6 krb5kdc[15347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:37:10.616926 15862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.572045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.10.247.193:35825)
0928 16:37:10.572297 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.572304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.597383 (+ 25079us) server_negotiation.cc:407] Connection header received
0928 16:37:10.597751 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.597759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.597869 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.597956 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:10.600147 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.601334 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.603250 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.603682 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.603786 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.609507 (+ 5721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:10.609525 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:10.609547 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:10.609606 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:10.612985 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.614172 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.614183 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.614186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.614411 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.614915 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.614920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.614922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.615466 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:10.615787 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.616263 (+ 476us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.616509 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260928 16:37:10.617678 15574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:10.746057 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 15452
I20260928 16:37:10.763742 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 15585
Sep 28 16:37:10 dist-test-slave-9fn6 krb5kdc[15347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:37:10.783268 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.764760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.10.247.194:43391)
0928 16:37:10.765009 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.765017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.765047 (+ 30us) server_negotiation.cc:407] Connection header received
0928 16:37:10.765141 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.765149 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.765275 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.765425 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:10.766730 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.767949 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.769358 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.770082 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.770233 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.774849 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:10.774878 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:10.774906 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:10.774973 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:10.779547 (+ 4574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.780489 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.780501 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.780505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.780848 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.781284 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.781290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.781293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.781914 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:10.782277 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.782587 (+ 310us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.782861 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260928 16:37:10.784133 15708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:10.879616 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 15585
I20260928 16:37:10.896817 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 15719
Sep 28 16:37:10 dist-test-slave-9fn6 krb5kdc[15347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:37:10.915419 15864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:10.897848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.10.247.195:43745)
0928 16:37:10.898054 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:10.898061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:10.898154 (+ 93us) server_negotiation.cc:407] Connection header received
0928 16:37:10.898418 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:10.898423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:10.898515 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:10.898625 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:10.900095 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.901070 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.902433 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:10.902910 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:10.903026 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:10.908316 (+ 5290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:10.908336 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:10.908356 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:10.908407 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:10.911778 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.912780 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.912791 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.912794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.913006 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:10.913511 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:10.913516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:10.913518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:10.914035 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:10.914330 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:10.914781 (+ 451us) server_negotiation.cc:299] Negotiation successful
0928 16:37:10.915041 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260928 16:37:10.916200 15841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:11.055349 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 15719
I20260928 16:37:11.075321 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 15360
I20260928 16:37:11.076882 15421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:11.204598 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 15360
2026-09-28T16:37:11Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5352 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-28T16:37:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:11Z Disabled control of system clock
I20260928 16:37:11.282902 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38519
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:38519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 16:37:11.664753 15899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:11.665117 15899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:11.665215 15899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:11.672833 15899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:11.672956 15899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:11.672994 15899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:11.673055 15899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:11.673105 15899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:11.682894 15899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:38519
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:11.684533 15899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:11.686757 15899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:11.700199 15904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:11.700145 15907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:11.700122 15905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:11.701108 15899 server_base.cc:1034] running on GCE node
I20260928 16:37:11.702066 15899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:11.703886 15899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:11.705114 15899 hybrid_clock.cc:648] HybridClock initialized: now 1790613431705060 us; error 72 us; skew 500 ppm
I20260928 16:37:11.705569 15899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:37:11.708376 15899 webserver.cc:460] Webserver started at http://127.10.247.254:41957/ using document root <none> and password file <none>
I20260928 16:37:11.709262 15899 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:11.709378 15899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:11.709730 15899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:11.713392 15899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "40c4738aa1c7423e9312b68d831b0412"
format_stamp: "Formatted at 2026-09-28 16:37:11 on dist-test-slave-9fn6"
server_key: "3be4e7cc525d80c6acb97ce39dd0020f"
server_key_iv: "00a1beb90d67cca17047d04e3062aa49"
server_key_version: "encryptionkey@0"
I20260928 16:37:11.714322 15899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "40c4738aa1c7423e9312b68d831b0412"
format_stamp: "Formatted at 2026-09-28 16:37:11 on dist-test-slave-9fn6"
server_key: "3be4e7cc525d80c6acb97ce39dd0020f"
server_key_iv: "00a1beb90d67cca17047d04e3062aa49"
server_key_version: "encryptionkey@0"
I20260928 16:37:11.720536 15899 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:37:11.724435 15913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:11.726140 15899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:37:11.726326 15899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "40c4738aa1c7423e9312b68d831b0412"
format_stamp: "Formatted at 2026-09-28 16:37:11 on dist-test-slave-9fn6"
server_key: "3be4e7cc525d80c6acb97ce39dd0020f"
server_key_iv: "00a1beb90d67cca17047d04e3062aa49"
server_key_version: "encryptionkey@0"
I20260928 16:37:11.726557 15899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:11.733424 15899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:11.734387 15899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:11.734702 15899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:11.753985 15899 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:38519
I20260928 16:37:11.754010 15964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:38519 every 8 connection(s)
I20260928 16:37:11.755753 15899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:11.759125 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 15899
I20260928 16:37:11.759308 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:11.760051 11231 external_mini_cluster.cc:1428] Setting key 11cecde67877aaec869356c9b7fa2825
I20260928 16:37:11.760669 15965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:11.770167 15968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:11.762445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.10.247.254:38519)
0928 16:37:11.763554 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:11.763576 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:11.763628 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:37:11.765907 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:11.765933 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:11.766445 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:11.766838 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:11.768088 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:11.768099 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:11.768117 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:11.768139 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:11.768229 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:11.768253 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:11.768534 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:11.768719 (+ 185us) server_negotiation.cc:299] Negotiation successful
0928 16:37:11.769082 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":380,"threads_started":1}
I20260928 16:37:11.775599 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412: Bootstrap starting.
I20260928 16:37:11.779366 15965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:11.780537 15965 log.cc:824] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:11.784209 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412: No bootstrap required, opened a new log
I20260928 16:37:11.792900 15965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c4738aa1c7423e9312b68d831b0412" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38519 } }
I20260928 16:37:11.793349 15965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:11.793452 15965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c4738aa1c7423e9312b68d831b0412, State: Initialized, Role: FOLLOWER
I20260928 16:37:11.794087 15965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c4738aa1c7423e9312b68d831b0412" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38519 } }
I20260928 16:37:11.794329 15965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:11.794466 15965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:11.794643 15965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:11.797544 15965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c4738aa1c7423e9312b68d831b0412" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38519 } }
I20260928 16:37:11.798280 15965 leader_election.cc:302] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40c4738aa1c7423e9312b68d831b0412; no voters:
I20260928 16:37:11.798959 15965 leader_election.cc:288] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:11.799249 15970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:11.800037 15970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [term 1 LEADER]: Becoming Leader. State: Replica: 40c4738aa1c7423e9312b68d831b0412, State: Running, Role: LEADER
I20260928 16:37:11.800705 15970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c4738aa1c7423e9312b68d831b0412" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38519 } }
I20260928 16:37:11.801429 15965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:11.803790 15971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40c4738aa1c7423e9312b68d831b0412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c4738aa1c7423e9312b68d831b0412" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38519 } } }
I20260928 16:37:11.804302 15971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:11.803790 15972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40c4738aa1c7423e9312b68d831b0412. Latest consensus state: current_term: 1 leader_uuid: "40c4738aa1c7423e9312b68d831b0412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c4738aa1c7423e9312b68d831b0412" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38519 } } }
I20260928 16:37:11.804499 15972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:11.808815 15975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:11.815218 15975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:11.825502 15975 catalog_manager.cc:1347] Generated new cluster ID: 3e7dd5eaacff4af3b02e281a9758ae99
I20260928 16:37:11.825656 15975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:11.846042 15975 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:11.846638 15975 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:11.858851 15975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412: Generated new TSK 0
I20260928 16:37:11.859989 15975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:37:11.873674 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38519
--builtin_ntp_servers=127.10.247.212:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 16:37:12.265699 15989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:12.266081 15989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:12.266180 15989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:12.274341 15989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:12.274499 15989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:12.274634 15989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:37:12.285334 15989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:12.287103 15989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:12.289414 15989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:12.305318 15995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:12.305403 15994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:12.305893 15997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:12.306855 15989 server_base.cc:1034] running on GCE node
I20260928 16:37:12.307632 15989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:12.309019 15989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:12.310241 15989 hybrid_clock.cc:648] HybridClock initialized: now 1790613432310170 us; error 55 us; skew 500 ppm
I20260928 16:37:12.310616 15989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:37:12.313294 15989 webserver.cc:460] Webserver started at http://127.10.247.193:40447/ using document root <none> and password file <none>
I20260928 16:37:12.314137 15989 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:12.314234 15989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:12.314590 15989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:12.318040 15989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "1f6375e2dc234abc9701e452b536d2ab"
format_stamp: "Formatted at 2026-09-28 16:37:12 on dist-test-slave-9fn6"
server_key: "7080ce88ed3d4ca42d674797d624aa3b"
server_key_iv: "7c08ecbc32094d9b4add42b3bd4fdf28"
server_key_version: "encryptionkey@0"
I20260928 16:37:12.318881 15989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "1f6375e2dc234abc9701e452b536d2ab"
format_stamp: "Formatted at 2026-09-28 16:37:12 on dist-test-slave-9fn6"
server_key: "7080ce88ed3d4ca42d674797d624aa3b"
server_key_iv: "7c08ecbc32094d9b4add42b3bd4fdf28"
server_key_version: "encryptionkey@0"
I20260928 16:37:12.324753 15989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 16:37:12.328691 16003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:12.330323 15989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:37:12.330502 15989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "1f6375e2dc234abc9701e452b536d2ab"
format_stamp: "Formatted at 2026-09-28 16:37:12 on dist-test-slave-9fn6"
server_key: "7080ce88ed3d4ca42d674797d624aa3b"
server_key_iv: "7c08ecbc32094d9b4add42b3bd4fdf28"
server_key_version: "encryptionkey@0"
I20260928 16:37:12.330745 15989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:12.337249 15989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:12.338151 15989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:12.338394 15989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:12.339385 15989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:12.341063 15989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:12.341152 15989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:12.341243 15989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:12.341297 15989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:12.375304 15989 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:37121
I20260928 16:37:12.375348 16115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:37121 every 8 connection(s)
I20260928 16:37:12.376968 15989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:12.383246 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 15989
I20260928 16:37:12.383456 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:37:12.384171 11231 external_mini_cluster.cc:1428] Setting key 5aaae4a2c717668e074d6dbdfc0e8011
I20260928 16:37:12.386977 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:12.380028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:41745 (local address 127.10.247.254:38519)
0928 16:37:12.380555 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:12.380561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:12.382004 (+ 1443us) server_negotiation.cc:407] Connection header received
0928 16:37:12.384119 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:12.384126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:12.384217 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:12.384339 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:12.386092 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:12.386099 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:12.386114 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:12.386138 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:12.386208 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:12.386225 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:12.386356 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:12.386558 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 16:37:12.386642 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
I20260928 16:37:12.388150 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38519
--builtin_ntp_servers=127.10.247.212:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260928 16:37:12.388516 16119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:12.380643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38519 (local address 127.10.247.193:41745)
0928 16:37:12.381774 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:12.381848 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:12.383781 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:12.384749 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:12.384764 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:12.384924 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:12.385712 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:37:12.385715 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:12.385766 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:37:12.385888 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:12.386326 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:12.386333 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:12.386339 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 16:37:12.386663 (+ 324us) client_negotiation.cc:240] Negotiation successful
0928 16:37:12.386984 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":349,"threads_started":1}
I20260928 16:37:12.390441 16116 heartbeater.cc:347] Connected to a master server at 127.10.247.254:38519
I20260928 16:37:12.390810 16116 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:12.391607 16116 heartbeater.cc:511] Master 127.10.247.254:38519 requested a full tablet report, sending...
I20260928 16:37:12.394531 15930 ts_manager.cc:194] Registered new tserver with Master: 1f6375e2dc234abc9701e452b536d2ab (127.10.247.193:37121)
W20260928 16:37:12.782459 16120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:12.782861 16120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:12.782990 16120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:12.791282 16120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:12.791440 16120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:12.791607 16120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:12.802386 16120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:12.804266 16120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:12.806816 16120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:12.823293 16125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:12.823192 16126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:12.823192 16128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:12.824486 16120 server_base.cc:1034] running on GCE node
I20260928 16:37:12.825390 16120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:12.826617 16120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:12.827840 16120 hybrid_clock.cc:648] HybridClock initialized: now 1790613432827782 us; error 54 us; skew 500 ppm
I20260928 16:37:12.828219 16120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:37:12.831313 16120 webserver.cc:460] Webserver started at http://127.10.247.194:44927/ using document root <none> and password file <none>
I20260928 16:37:12.832353 16120 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:12.832489 16120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:12.832949 16120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:12.836933 16120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "cb89586a9bd045669754a88f61f01778"
format_stamp: "Formatted at 2026-09-28 16:37:12 on dist-test-slave-9fn6"
server_key: "f76b3a72f9200eb808cd2425f7b832ee"
server_key_iv: "e766b103afd9f6ed4c358689bc83237b"
server_key_version: "encryptionkey@0"
I20260928 16:37:12.837935 16120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "cb89586a9bd045669754a88f61f01778"
format_stamp: "Formatted at 2026-09-28 16:37:12 on dist-test-slave-9fn6"
server_key: "f76b3a72f9200eb808cd2425f7b832ee"
server_key_iv: "e766b103afd9f6ed4c358689bc83237b"
server_key_version: "encryptionkey@0"
I20260928 16:37:12.844551 16120 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260928 16:37:12.849478 16134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:12.851413 16120 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:37:12.851641 16120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "cb89586a9bd045669754a88f61f01778"
format_stamp: "Formatted at 2026-09-28 16:37:12 on dist-test-slave-9fn6"
server_key: "f76b3a72f9200eb808cd2425f7b832ee"
server_key_iv: "e766b103afd9f6ed4c358689bc83237b"
server_key_version: "encryptionkey@0"
I20260928 16:37:12.851929 16120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:12.859509 16120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:12.860638 16120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:12.860947 16120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:12.862156 16120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:12.864045 16120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:12.864158 16120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:12.864286 16120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:12.864364 16120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:12.904044 16120 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:45887
I20260928 16:37:12.904106 16246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:45887 every 8 connection(s)
I20260928 16:37:12.905753 16120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:12.907388 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 16120
I20260928 16:37:12.907574 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:37:12.908125 11231 external_mini_cluster.cc:1428] Setting key dd411058d30a249222e70e0fdd9218c4
I20260928 16:37:12.912286 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38519
--builtin_ntp_servers=127.10.247.212:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260928 16:37:12.916428 16249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:12.908928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:42441 (local address 127.10.247.254:38519)
0928 16:37:12.911731 (+ 2803us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:12.911737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:12.911760 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:37:12.913740 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:12.913747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:12.913830 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:12.913960 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:12.915587 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:12.915593 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:12.915597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:12.915619 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:12.915682 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:12.915699 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:12.915840 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:12.915998 (+ 158us) server_negotiation.cc:299] Negotiation successful
0928 16:37:12.916081 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2666,"thread_start_us":2526,"threads_started":1}
I20260928 16:37:12.917693 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:12.909743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38519 (local address 127.10.247.194:42441)
0928 16:37:12.911308 (+ 1565us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:12.911386 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:12.913380 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:12.914253 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:12.914263 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:12.914400 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:12.915212 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:37:12.915215 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:12.915265 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:37:12.915386 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:12.915797 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:12.915805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:12.915811 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 16:37:12.916076 (+ 265us) client_negotiation.cc:240] Negotiation successful
0928 16:37:12.916358 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":398,"threads_started":1}
I20260928 16:37:12.919479 16247 heartbeater.cc:347] Connected to a master server at 127.10.247.254:38519
I20260928 16:37:12.919842 16247 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:12.920594 16247 heartbeater.cc:511] Master 127.10.247.254:38519 requested a full tablet report, sending...
I20260928 16:37:12.922472 15930 ts_manager.cc:194] Registered new tserver with Master: cb89586a9bd045669754a88f61f01778 (127.10.247.194:45887)
W20260928 16:37:13.307569 16251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:13.307963 16251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:13.308079 16251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:13.316753 16251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:13.316887 16251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:13.317060 16251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:13.327538 16251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:13.329407 16251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:13.331777 16251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:13.347343 16257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:13.347308 16256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:13.347270 16259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:13.348189 16251 server_base.cc:1034] running on GCE node
I20260928 16:37:13.349036 16251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:13.350586 16251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:13.351853 16251 hybrid_clock.cc:648] HybridClock initialized: now 1790613433351780 us; error 59 us; skew 500 ppm
I20260928 16:37:13.352321 16251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:37:13.359104 16251 webserver.cc:460] Webserver started at http://127.10.247.195:34887/ using document root <none> and password file <none>
I20260928 16:37:13.360248 16251 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:13.360387 16251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:13.360837 16251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:13.364560 16251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "dc8a6b9db8ac419884eaeb701bd26b17"
format_stamp: "Formatted at 2026-09-28 16:37:13 on dist-test-slave-9fn6"
server_key: "ab94c5beac66b29ae5b0b27863cbaa0b"
server_key_iv: "c24c3f8bdd3ec84d4cffbc3059e6b52a"
server_key_version: "encryptionkey@0"
I20260928 16:37:13.365680 16251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "dc8a6b9db8ac419884eaeb701bd26b17"
format_stamp: "Formatted at 2026-09-28 16:37:13 on dist-test-slave-9fn6"
server_key: "ab94c5beac66b29ae5b0b27863cbaa0b"
server_key_iv: "c24c3f8bdd3ec84d4cffbc3059e6b52a"
server_key_version: "encryptionkey@0"
I20260928 16:37:13.372334 16251 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:37:13.377023 16265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:13.378826 16251 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:37:13.379026 16251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "dc8a6b9db8ac419884eaeb701bd26b17"
format_stamp: "Formatted at 2026-09-28 16:37:13 on dist-test-slave-9fn6"
server_key: "ab94c5beac66b29ae5b0b27863cbaa0b"
server_key_iv: "c24c3f8bdd3ec84d4cffbc3059e6b52a"
server_key_version: "encryptionkey@0"
I20260928 16:37:13.379348 16251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:13.386094 16251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:13.387145 16251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:13.387446 16251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:13.388630 16251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:13.390465 16251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:13.390571 16251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:13.390698 16251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:13.390772 16251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:13.398970 16116 heartbeater.cc:503] Master 127.10.247.254:38519 was elected leader, sending a full tablet report...
I20260928 16:37:13.425179 16251 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:36743
I20260928 16:37:13.425215 16378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:36743 every 8 connection(s)
I20260928 16:37:13.426786 16251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:13.431496 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 16251
I20260928 16:37:13.431727 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:37:13.432384 11231 external_mini_cluster.cc:1428] Setting key 81beef94864c98b0cf9a985249e18021
I20260928 16:37:13.437507 16381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.430031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:49049 (local address 127.10.247.254:38519)
0928 16:37:13.430635 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:13.430641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:13.432244 (+ 1603us) server_negotiation.cc:407] Connection header received
0928 16:37:13.434547 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:13.434553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:13.434649 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:13.434758 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:13.436596 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:13.436601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:13.436604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:13.436661 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:13.436716 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:13.436730 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:13.436835 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:13.437054 (+ 219us) server_negotiation.cc:299] Negotiation successful
0928 16:37:13.437136 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":291,"threads_started":1}
I20260928 16:37:13.438657 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.430747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38519 (local address 127.10.247.195:49049)
0928 16:37:13.432030 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:13.432090 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:13.434223 (+ 2133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:13.435297 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:13.435309 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:13.435456 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:13.436264 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:37:13.436267 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:13.436313 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:37:13.436421 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:13.436832 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:13.436844 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:13.436853 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 16:37:13.437149 (+ 296us) client_negotiation.cc:240] Negotiation successful
0928 16:37:13.437479 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":381,"threads_started":1}
I20260928 16:37:13.440143 16379 heartbeater.cc:347] Connected to a master server at 127.10.247.254:38519
I20260928 16:37:13.440495 16379 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:13.441267 16379 heartbeater.cc:511] Master 127.10.247.254:38519 requested a full tablet report, sending...
I20260928 16:37:13.442848 15930 ts_manager.cc:194] Registered new tserver with Master: dc8a6b9db8ac419884eaeb701bd26b17 (127.10.247.195:36743)
I20260928 16:37:13.448954 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:37:13.456866 16381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.454047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56902 (local address 127.10.247.254:38519)
0928 16:37:13.454255 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:13.454262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:13.454846 (+ 584us) server_negotiation.cc:407] Connection header received
0928 16:37:13.455065 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:13.455072 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:13.455164 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:13.455275 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:13.456240 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:13.456245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:13.456249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:13.456267 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:13.456308 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:13.456322 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:13.456428 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:13.456469 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 16:37:13.456547 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:37:13.461648 15930 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:37:13.469584 15930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:37:13.496690 16392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.484217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:45887 (local address 127.0.0.1:52134)
0928 16:37:13.490910 (+ 6693us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:13.490974 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:13.492434 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:13.493849 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:13.493859 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:13.493973 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:13.495042 (+ 1069us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:37:13.495045 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:13.495079 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:37:13.495202 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:13.495784 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:13.495796 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:13.495804 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 16:37:13.495981 (+ 177us) client_negotiation.cc:240] Negotiation successful
0928 16:37:13.496091 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6296,"thread_start_us":6048,"threads_started":1}
I20260928 16:37:13.497534 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.484413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.10.247.194:45887)
0928 16:37:13.492632 (+ 8219us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:13.492645 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:13.492688 (+ 43us) server_negotiation.cc:407] Connection header received
0928 16:37:13.492785 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:13.492795 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:13.493146 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:13.493336 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:13.495412 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:13.495423 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:13.495440 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:13.495484 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:13.495562 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:13.495590 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:13.495732 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:13.496985 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0928 16:37:13.497132 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7970,"thread_start_us":7907,"threads_started":1}
I20260928 16:37:13.498330 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.484518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:36743 (local address 127.0.0.1:43716)
0928 16:37:13.493072 (+ 8554us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:13.493090 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:13.493201 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:13.494775 (+ 1574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:13.494781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:13.494812 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:13.495581 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:37:13.495585 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:13.495605 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:37:13.495658 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:13.497615 (+ 1957us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:13.497622 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:13.497628 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 16:37:13.497767 (+ 139us) client_negotiation.cc:240] Negotiation successful
0928 16:37:13.497831 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8398,"thread_start_us":8261,"threads_started":1}
I20260928 16:37:13.500794 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.485088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.10.247.195:36743)
0928 16:37:13.493854 (+ 8766us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:13.493863 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:13.493900 (+ 37us) server_negotiation.cc:407] Connection header received
0928 16:37:13.493995 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:13.494004 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:13.494255 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:13.495390 (+ 1135us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:13.497352 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:13.497362 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:13.497378 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:13.497397 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:13.497453 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:13.497475 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:13.500158 (+ 2683us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:13.500241 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 16:37:13.500369 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8577,"thread_start_us":8384,"threads_started":1}
I20260928 16:37:13.502357 16392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.496258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37121 (local address 127.0.0.1:43054)
0928 16:37:13.498246 (+ 1988us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:13.498267 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:13.498357 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:13.499742 (+ 1385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:13.499746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:13.499795 (+ 49us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:13.500389 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:37:13.500392 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:13.500410 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:37:13.500455 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:13.501985 (+ 1530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:13.501990 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:13.501995 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 16:37:13.502099 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 16:37:13.502156 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1861,"thread_start_us":3308,"threads_started":1}
I20260928 16:37:13.502691 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.493698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.10.247.193:37121)
0928 16:37:13.499141 (+ 5443us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:13.499162 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:13.499194 (+ 32us) server_negotiation.cc:407] Connection header received
0928 16:37:13.499273 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:13.499282 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:13.499545 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:13.500133 (+ 588us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:13.501745 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:13.501753 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:13.501768 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:13.501786 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:13.501842 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:13.501864 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:13.502046 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:13.502117 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 16:37:13.502231 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5248,"thread_start_us":5070,"threads_started":1}
I20260928 16:37:13.503685 16182 tablet_service.cc:1467] Processing CreateTablet for tablet 20d45ead3aac466d84324c79113bfe95 (DEFAULT_TABLE table=test-table [id=37086c7a61cc41ddab5c8474b7a18e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:13.506175 16182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d45ead3aac466d84324c79113bfe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:13.505808 16314 tablet_service.cc:1467] Processing CreateTablet for tablet 20d45ead3aac466d84324c79113bfe95 (DEFAULT_TABLE table=test-table [id=37086c7a61cc41ddab5c8474b7a18e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:13.506579 16051 tablet_service.cc:1467] Processing CreateTablet for tablet 20d45ead3aac466d84324c79113bfe95 (DEFAULT_TABLE table=test-table [id=37086c7a61cc41ddab5c8474b7a18e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:13.507354 16314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d45ead3aac466d84324c79113bfe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:13.507990 16051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d45ead3aac466d84324c79113bfe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:13.520992 16398 tablet_bootstrap.cc:492] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: Bootstrap starting.
I20260928 16:37:13.523464 16400 tablet_bootstrap.cc:492] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: Bootstrap starting.
I20260928 16:37:13.526736 16398 tablet_bootstrap.cc:654] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:13.526986 16399 tablet_bootstrap.cc:492] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: Bootstrap starting.
I20260928 16:37:13.527534 16400 tablet_bootstrap.cc:654] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:13.528995 16398 log.cc:824] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:13.529004 16400 log.cc:824] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:13.530454 16399 tablet_bootstrap.cc:654] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:13.531930 16399 log.cc:824] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:13.532848 16400 tablet_bootstrap.cc:492] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: No bootstrap required, opened a new log
I20260928 16:37:13.532847 16398 tablet_bootstrap.cc:492] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: No bootstrap required, opened a new log
I20260928 16:37:13.533188 16400 ts_tablet_manager.cc:1397] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260928 16:37:13.533200 16398 ts_tablet_manager.cc:1397] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260928 16:37:13.535756 16399 tablet_bootstrap.cc:492] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: No bootstrap required, opened a new log
I20260928 16:37:13.536203 16399 ts_tablet_manager.cc:1397] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260928 16:37:13.542281 16398 raft_consensus.cc:348] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } }
I20260928 16:37:13.542789 16398 raft_consensus.cc:374] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:13.542912 16398 raft_consensus.cc:729] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc8a6b9db8ac419884eaeb701bd26b17, State: Initialized, Role: FOLLOWER
I20260928 16:37:13.543709 16398 consensus_queue.cc:260] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } }
I20260928 16:37:13.544517 16400 raft_consensus.cc:348] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } }
I20260928 16:37:13.545071 16400 raft_consensus.cc:374] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:13.545167 16400 raft_consensus.cc:729] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb89586a9bd045669754a88f61f01778, State: Initialized, Role: FOLLOWER
I20260928 16:37:13.545214 16379 heartbeater.cc:503] Master 127.10.247.254:38519 was elected leader, sending a full tablet report...
I20260928 16:37:13.546041 16398 ts_tablet_manager.cc:1428] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260928 16:37:13.545890 16400 consensus_queue.cc:260] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } }
I20260928 16:37:13.547684 16399 raft_consensus.cc:348] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } }
I20260928 16:37:13.548266 16399 raft_consensus.cc:374] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:13.548358 16399 raft_consensus.cc:729] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f6375e2dc234abc9701e452b536d2ab, State: Initialized, Role: FOLLOWER
I20260928 16:37:13.549252 16399 consensus_queue.cc:260] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } }
I20260928 16:37:13.549566 16247 heartbeater.cc:503] Master 127.10.247.254:38519 was elected leader, sending a full tablet report...
I20260928 16:37:13.551425 16399 ts_tablet_manager.cc:1428] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260928 16:37:13.552554 16400 ts_tablet_manager.cc:1428] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
W20260928 16:37:13.630039 16117 tablet.cc:2369] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:37:13.658140 16248 tablet.cc:2369] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:37:13.678790 16380 tablet.cc:2369] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:37:13.790153 16406 raft_consensus.cc:482] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:13.790577 16406 raft_consensus.cc:504] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } }
I20260928 16:37:13.792883 16406 leader_election.cc:288] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb89586a9bd045669754a88f61f01778 (127.10.247.194:45887), dc8a6b9db8ac419884eaeb701bd26b17 (127.10.247.195:36743)
I20260928 16:37:13.797046 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.793496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:46055 (local address 127.10.247.194:45887)
0928 16:37:13.793706 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:13.793714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:13.794635 (+ 921us) server_negotiation.cc:407] Connection header received
0928 16:37:13.794827 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:13.794834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:13.794918 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:13.795029 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:13.796232 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:13.796238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:13.796242 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:13.796265 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:13.796312 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:13.796329 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:13.796457 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:13.796581 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 16:37:13.796717 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 16:37:13.797073 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.793570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:45115 (local address 127.10.247.195:36743)
0928 16:37:13.793782 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:13.793788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:13.794714 (+ 926us) server_negotiation.cc:407] Connection header received
0928 16:37:13.794886 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:13.794891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:13.794969 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:13.795072 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:13.796280 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:13.796287 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:13.796292 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:13.796311 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:13.796352 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:13.796368 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:13.796497 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:13.796745 (+ 248us) server_negotiation.cc:299] Negotiation successful
0928 16:37:13.796817 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 16:37:13.797122 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.793496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:45887 (local address 127.10.247.193:46055)
0928 16:37:13.794493 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:13.794512 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:13.794657 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:13.795087 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:13.795091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:13.795116 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:13.795979 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:37:13.795983 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:13.796005 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:37:13.796066 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:13.796439 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:13.796445 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:13.796450 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 16:37:13.796634 (+ 184us) client_negotiation.cc:240] Negotiation successful
0928 16:37:13.796713 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"thread_start_us":488,"threads_started":1}
I20260928 16:37:13.797830 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.793514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:36743 (local address 127.10.247.193:45115)
0928 16:37:13.794595 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:13.794621 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:13.794756 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:13.795261 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:13.795265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:13.795287 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:13.796105 (+ 818us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:37:13.796108 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:13.796121 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:37:13.796156 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:13.796553 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:13.796557 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:13.796562 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 16:37:13.797338 (+ 776us) client_negotiation.cc:240] Negotiation successful
0928 16:37:13.797394 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":220,"thread_start_us":509,"threads_started":1}
I20260928 16:37:13.798292 16202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d45ead3aac466d84324c79113bfe95" candidate_uuid: "1f6375e2dc234abc9701e452b536d2ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb89586a9bd045669754a88f61f01778" is_pre_election: true
I20260928 16:37:13.798955 16334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d45ead3aac466d84324c79113bfe95" candidate_uuid: "1f6375e2dc234abc9701e452b536d2ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" is_pre_election: true
I20260928 16:37:13.799136 16202 raft_consensus.cc:2454] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f6375e2dc234abc9701e452b536d2ab in term 0.
I20260928 16:37:13.799626 16334 raft_consensus.cc:2454] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f6375e2dc234abc9701e452b536d2ab in term 0.
I20260928 16:37:13.800019 16007 leader_election.cc:302] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f6375e2dc234abc9701e452b536d2ab, cb89586a9bd045669754a88f61f01778; no voters:
I20260928 16:37:13.800487 16406 raft_consensus.cc:2781] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:13.800678 16406 raft_consensus.cc:482] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:13.800778 16406 raft_consensus.cc:3037] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:13.803579 16406 raft_consensus.cc:504] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } }
I20260928 16:37:13.804421 16406 leader_election.cc:288] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [CANDIDATE]: Term 1 election: Requested vote from peers cb89586a9bd045669754a88f61f01778 (127.10.247.194:45887), dc8a6b9db8ac419884eaeb701bd26b17 (127.10.247.195:36743)
I20260928 16:37:13.805285 16334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d45ead3aac466d84324c79113bfe95" candidate_uuid: "1f6375e2dc234abc9701e452b536d2ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc8a6b9db8ac419884eaeb701bd26b17"
I20260928 16:37:13.805285 16202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d45ead3aac466d84324c79113bfe95" candidate_uuid: "1f6375e2dc234abc9701e452b536d2ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb89586a9bd045669754a88f61f01778"
I20260928 16:37:13.805591 16202 raft_consensus.cc:3037] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:13.805591 16334 raft_consensus.cc:3037] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:13.809459 16334 raft_consensus.cc:2454] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f6375e2dc234abc9701e452b536d2ab in term 1.
I20260928 16:37:13.809459 16202 raft_consensus.cc:2454] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f6375e2dc234abc9701e452b536d2ab in term 1.
I20260928 16:37:13.810199 16007 leader_election.cc:302] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f6375e2dc234abc9701e452b536d2ab, cb89586a9bd045669754a88f61f01778, dc8a6b9db8ac419884eaeb701bd26b17; no voters:
I20260928 16:37:13.810674 16406 raft_consensus.cc:2781] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:13.811206 16406 raft_consensus.cc:686] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 1 LEADER]: Becoming Leader. State: Replica: 1f6375e2dc234abc9701e452b536d2ab, State: Running, Role: LEADER
I20260928 16:37:13.811923 16406 consensus_queue.cc:237] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } }
I20260928 16:37:13.818230 15930 catalog_manager.cc:5495] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f6375e2dc234abc9701e452b536d2ab (127.10.247.193). New cstate: current_term: 1 leader_uuid: "1f6375e2dc234abc9701e452b536d2ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f6375e2dc234abc9701e452b536d2ab" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37121 } health_report { overall_health: HEALTHY } } }
I20260928 16:37:13.880764 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.878072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.10.247.193:37121)
0928 16:37:13.878287 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:13.878292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:13.878316 (+ 24us) server_negotiation.cc:407] Connection header received
0928 16:37:13.878651 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:13.878657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:13.878778 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:13.878882 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:13.879983 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:13.879988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:13.879992 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:13.880010 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:13.880055 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:13.880070 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:13.880200 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:13.880371 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 16:37:13.880443 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260928 16:37:13.892305 16202 raft_consensus.cc:1261] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [term 1 FOLLOWER]: Refusing update from remote peer 1f6375e2dc234abc9701e452b536d2ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:13.892306 16334 raft_consensus.cc:1261] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [term 1 FOLLOWER]: Refusing update from remote peer 1f6375e2dc234abc9701e452b536d2ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:13.893843 16406 consensus_queue.cc:1035] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc8a6b9db8ac419884eaeb701bd26b17" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:13.894316 16409 consensus_queue.cc:1035] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb89586a9bd045669754a88f61f01778" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 45887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:13.924845 16413 mvcc.cc:205] Tried to move back new op lower bound from 7334352625207414784 to 7334352624898994176. Current Snapshot: MvccSnapshot[applied={T|T < 7334352625207414784 or (T in {7334352625207414784})}]
I20260928 16:37:13.940728 15928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:13.941105 15928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:13.946498 15928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40c4738aa1c7423e9312b68d831b0412: Sending DeleteTablet for 3 replicas of tablet 20d45ead3aac466d84324c79113bfe95
I20260928 16:37:13.948104 16314 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d45ead3aac466d84324c79113bfe95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:13 UTC) from {username='slave'} at 127.0.0.1:43716
I20260928 16:37:13.948086 16182 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d45ead3aac466d84324c79113bfe95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:13 UTC) from {username='slave'} at 127.0.0.1:52134
I20260928 16:37:13.948758 16051 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d45ead3aac466d84324c79113bfe95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:13 UTC) from {username='slave'} at 127.0.0.1:43054
I20260928 16:37:13.957583 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 15989
I20260928 16:37:13.958812 16424 tablet_replica.cc:330] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: stopping tablet replica
I20260928 16:37:13.959043 16423 tablet_replica.cc:330] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: stopping tablet replica
I20260928 16:37:13.959695 16423 raft_consensus.cc:2229] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:13.959942 16424 raft_consensus.cc:2229] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:37:13.960424 16423 raft_consensus.cc:2258] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:13.961187 16424 raft_consensus.cc:2258] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:13.963835 16423 ts_tablet_manager.cc:1905] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:13.963840 16424 ts_tablet_manager.cc:1905] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:13.964886 16422 tablet_replica.cc:330] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: stopping tablet replica
I20260928 16:37:13.965570 16422 raft_consensus.cc:2229] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:13.966248 16422 raft_consensus.cc:2258] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:13.973413 16424 ts_tablet_manager.cc:1918] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:13.973755 16424 log.cc:1196] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/20d45ead3aac466d84324c79113bfe95
I20260928 16:37:13.973881 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:13.959027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43072 (local address 127.10.247.193:37121)
0928 16:37:13.959235 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:13.959242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:13.970409 (+ 11167us) server_negotiation.cc:407] Connection header received
0928 16:37:13.971184 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:13.971192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:13.971291 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:13.971383 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:13.972902 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:13.972910 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:13.972915 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:13.972943 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:13.973002 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:13.973021 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:13.973160 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:13.973444 (+ 284us) server_negotiation.cc:299] Negotiation successful
0928 16:37:13.973528 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 16:37:13.974789 16424 ts_tablet_manager.cc:1939] T 20d45ead3aac466d84324c79113bfe95 P 1f6375e2dc234abc9701e452b536d2ab: Deleting consensus metadata
I20260928 16:37:13.975178 16111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:13.975965 16422 ts_tablet_manager.cc:1905] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:13.978715 16423 ts_tablet_manager.cc:1918] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:13.979661 16423 log.cc:1196] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/20d45ead3aac466d84324c79113bfe95
I20260928 16:37:13.980494 16423 ts_tablet_manager.cc:1939] T 20d45ead3aac466d84324c79113bfe95 P cb89586a9bd045669754a88f61f01778: Deleting consensus metadata
I20260928 16:37:13.983309 15917 catalog_manager.cc:4841] TS cb89586a9bd045669754a88f61f01778 (127.10.247.194:45887): tablet 20d45ead3aac466d84324c79113bfe95 (table test-table [id=37086c7a61cc41ddab5c8474b7a18e88]) successfully deleted
I20260928 16:37:13.985515 16422 ts_tablet_manager.cc:1918] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:13.985746 16422 log.cc:1196] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/20d45ead3aac466d84324c79113bfe95
I20260928 16:37:13.986322 16422 ts_tablet_manager.cc:1939] T 20d45ead3aac466d84324c79113bfe95 P dc8a6b9db8ac419884eaeb701bd26b17: Deleting consensus metadata
I20260928 16:37:13.988543 15915 catalog_manager.cc:4841] TS dc8a6b9db8ac419884eaeb701bd26b17 (127.10.247.195:36743): tablet 20d45ead3aac466d84324c79113bfe95 (table test-table [id=37086c7a61cc41ddab5c8474b7a18e88]) successfully deleted
I20260928 16:37:14.087077 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 15989
I20260928 16:37:14.087234 15915 catalog_manager.cc:4841] TS 1f6375e2dc234abc9701e452b536d2ab (127.10.247.193:37121): tablet 20d45ead3aac466d84324c79113bfe95 (table test-table [id=37086c7a61cc41ddab5c8474b7a18e88]) successfully deleted
I20260928 16:37:14.105304 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 16120
I20260928 16:37:14.109041 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:14.106304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.10.247.194:45887)
0928 16:37:14.106512 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:14.106517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:14.106593 (+ 76us) server_negotiation.cc:407] Connection header received
0928 16:37:14.106836 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:14.106841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:14.107003 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:14.107111 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:14.108265 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:14.108271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:14.108274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:14.108295 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:14.108350 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:14.108366 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:14.108512 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:14.108557 (+ 45us) server_negotiation.cc:299] Negotiation successful
0928 16:37:14.108709 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 16:37:14.109910 16242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:14.207235 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 16120
I20260928 16:37:14.224305 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 16251
I20260928 16:37:14.228011 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:14.225550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.10.247.195:36743)
0928 16:37:14.225775 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:14.225780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:14.225804 (+ 24us) server_negotiation.cc:407] Connection header received
0928 16:37:14.225933 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:14.225937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:14.226107 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:14.226210 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:14.227267 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:14.227272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:14.227276 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:14.227293 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:14.227334 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:14.227347 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:14.227473 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:14.227593 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 16:37:14.227654 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 16:37:14.228780 16374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:14.325587 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 16251
I20260928 16:37:14.342623 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 15899
I20260928 16:37:14.344254 15960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:14.465580 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 15899
2026-09-28T16:37:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3258 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260928 16:37:14.508304 11231 mini_oidc.cc:152] Starting JWKS server
I20260928 16:37:14.511499 11231 webserver.cc:460] Webserver started at https://127.0.0.1:46499/ using document root <none> and password file <none>
I20260928 16:37:14.511844 11231 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 16:37:14.512565 11231 webserver.cc:460] Webserver started at http://127.0.0.1:35849/ using document root <none> and password file <none>
2026-09-28T16:37:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:14Z Disabled control of system clock
I20260928 16:37:14.548429 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38409
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:38409
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46499/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/testchainca.cert with env {}
W20260928 16:37:14.920274 16435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:14.920749 16435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:14.920926 16435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:14.929229 16435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:14.929419 16435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:14.929494 16435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:14.929589 16435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:14.929670 16435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 16:37:14.929772 16435 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 16:37:14.929837 16435 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46499/jwks.json
I20260928 16:37:14.941696 16435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:38409
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46499/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:14.943589 16435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:14.945385 16435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:14.960780 16440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:14.961225 16443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:14.961284 16441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:14.962322 16435 server_base.cc:1034] running on GCE node
I20260928 16:37:14.963359 16435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:14.965485 16435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:14.966755 16435 hybrid_clock.cc:648] HybridClock initialized: now 1790613434966700 us; error 51 us; skew 500 ppm
I20260928 16:37:14.967233 16435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:37:14.970364 16435 webserver.cc:460] Webserver started at http://127.10.247.254:38897/ using document root <none> and password file <none>
I20260928 16:37:14.971274 16435 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:14.971405 16435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:14.971789 16435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:14.975557 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "0552db4fbc5248c0ba27c4688ee2113b"
format_stamp: "Formatted at 2026-09-28 16:37:14 on dist-test-slave-9fn6"
server_key: "fa67c2db7e1a93786dbaf1259bcfb2a2"
server_key_iv: "30f6e6c5a29d1f9b1317a3f7eb11319a"
server_key_version: "encryptionkey@0"
I20260928 16:37:14.976549 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "0552db4fbc5248c0ba27c4688ee2113b"
format_stamp: "Formatted at 2026-09-28 16:37:14 on dist-test-slave-9fn6"
server_key: "fa67c2db7e1a93786dbaf1259bcfb2a2"
server_key_iv: "30f6e6c5a29d1f9b1317a3f7eb11319a"
server_key_version: "encryptionkey@0"
I20260928 16:37:14.983116 16435 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260928 16:37:14.987455 16449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:14.989274 16435 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260928 16:37:14.989509 16435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "0552db4fbc5248c0ba27c4688ee2113b"
format_stamp: "Formatted at 2026-09-28 16:37:14 on dist-test-slave-9fn6"
server_key: "fa67c2db7e1a93786dbaf1259bcfb2a2"
server_key_iv: "30f6e6c5a29d1f9b1317a3f7eb11319a"
server_key_version: "encryptionkey@0"
I20260928 16:37:14.989773 16435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:15.003196 16435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:15.004383 16435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:15.004803 16435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:15.026530 16435 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:38409
I20260928 16:37:15.026542 16500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:38409 every 8 connection(s)
I20260928 16:37:15.028443 16435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:15.033725 16501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:15.034125 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 16435
I20260928 16:37:15.034396 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:15.035290 11231 external_mini_cluster.cc:1428] Setting key d04de8f15430b9524790db0fb1e59888
I20260928 16:37:15.050732 16501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b: Bootstrap starting.
I20260928 16:37:15.053126 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:15.037886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60086 (local address 127.10.247.254:38409)
0928 16:37:15.039311 (+ 1425us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:15.039325 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:15.039391 (+ 66us) server_negotiation.cc:407] Connection header received
0928 16:37:15.041552 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:15.041589 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:15.042342 (+ 753us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:15.042800 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:15.043591 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:15.045185 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:15.047323 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:15.048243 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:15.048415 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:15.050000 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:15.050010 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:15.050031 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:15.050747 (+ 716us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:15.050863 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:15.050897 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:15.051034 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:15.051335 (+ 301us) server_negotiation.cc:299] Negotiation successful
0928 16:37:15.051769 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":503,"threads_started":1}
I20260928 16:37:15.054927 16501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:15.056258 16501 log.cc:824] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:15.058099 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:37:15.059995 16501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b: No bootstrap required, opened a new log
I20260928 16:37:15.068849 16501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0552db4fbc5248c0ba27c4688ee2113b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38409 } }
I20260928 16:37:15.069384 16501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:15.069492 16501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0552db4fbc5248c0ba27c4688ee2113b, State: Initialized, Role: FOLLOWER
I20260928 16:37:15.070204 16501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0552db4fbc5248c0ba27c4688ee2113b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38409 } }
I20260928 16:37:15.070443 16501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:15.070583 16501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:15.070746 16501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:15.073271 16501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0552db4fbc5248c0ba27c4688ee2113b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38409 } }
I20260928 16:37:15.073860 16501 leader_election.cc:302] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0552db4fbc5248c0ba27c4688ee2113b; no voters:
I20260928 16:37:15.074540 16501 leader_election.cc:288] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:15.074697 16506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:15.075894 16506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [term 1 LEADER]: Becoming Leader. State: Replica: 0552db4fbc5248c0ba27c4688ee2113b, State: Running, Role: LEADER
I20260928 16:37:15.076524 16506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0552db4fbc5248c0ba27c4688ee2113b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38409 } }
I20260928 16:37:15.077261 16501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:15.079666 16507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0552db4fbc5248c0ba27c4688ee2113b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0552db4fbc5248c0ba27c4688ee2113b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38409 } } }
I20260928 16:37:15.080202 16507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:15.079640 16508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0552db4fbc5248c0ba27c4688ee2113b. Latest consensus state: current_term: 1 leader_uuid: "0552db4fbc5248c0ba27c4688ee2113b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0552db4fbc5248c0ba27c4688ee2113b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38409 } } }
I20260928 16:37:15.080318 16508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:15.081161 16512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:15.090308 16512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:15.102960 16512 catalog_manager.cc:1347] Generated new cluster ID: ad1145453c8a47cb9f40f0f22357aa41
I20260928 16:37:15.103147 16512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:15.121076 16512 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:15.122761 16512 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:15.133091 16512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0552db4fbc5248c0ba27c4688ee2113b: Generated new TSK 0
I20260928 16:37:15.134297 16512 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 16:37:15.183457 16531 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260928 16:37:15.183840 16531 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 16:37:15.186609 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:15.160546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.10.247.254:38409)
0928 16:37:15.160851 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:15.160859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:15.161211 (+ 352us) server_negotiation.cc:407] Connection header received
0928 16:37:15.161603 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:15.161612 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:15.161744 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:15.161959 (+ 215us) server_negotiation.cc:226] Negotiated authn=JWT
0928 16:37:15.162541 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:15.163778 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:15.165575 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:15.166373 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:15.166530 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:15.167157 (+ 627us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 16:37:15.185620 (+ 18463us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 16:37:15.186197 (+ 577us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60102: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":134}
W20260928 16:37:15.186779 16504 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60102: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260928 16:37:15.188552 16530 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:15.160351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38409 (local address 127.0.0.1:60102)
0928 16:37:15.161089 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:15.161108 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:15.161290 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:15.161994 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:15.161998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:15.162008 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0928 16:37:15.162369 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:15.162378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:15.163961 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:15.163967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:15.165396 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:15.165412 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:15.166576 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:15.166581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:15.166722 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:15.166802 (+ 80us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 16:37:15.186075 (+ 19273us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 16:37:15.186460 (+ 385us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.247.254:38409: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":358,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.247.254:38409: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46499/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260928 16:37:15.190599 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 16435
I20260928 16:37:15.192090 16496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:15.297103 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 16435
2026-09-28T16:37:15Z chronyd exiting
I20260928 16:37:15.348546 11231 test_util.cc:175] -----------------------------------------------
I20260928 16:37:15.348785 11231 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790613388455587-11231-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (845 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260928 16:37:15.354239 11231 mini_oidc.cc:152] Starting JWKS server
I20260928 16:37:15.356437 11231 webserver.cc:460] Webserver started at https://127.0.0.1:36523/ using document root <none> and password file <none>
I20260928 16:37:15.356734 11231 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 16:37:15.357509 11231 webserver.cc:460] Webserver started at http://127.0.0.1:41141/ using document root <none> and password file <none>
2026-09-28T16:37:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:15Z Disabled control of system clock
I20260928 16:37:15.393235 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37773
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:39545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37773
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36523/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/testchainca.cert with env {}
W20260928 16:37:15.772707 16538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:15.773118 16538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:15.773280 16538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:15.781332 16538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:15.781499 16538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:15.781565 16538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:15.781654 16538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:15.781724 16538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 16:37:15.781819 16538 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 16:37:15.781876 16538 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36523/jwks.json
I20260928 16:37:15.793407 16538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36523/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:15.795249 16538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:15.797012 16538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:15.810947 16543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:15.811127 16544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:15.811160 16546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:15.811899 16538 server_base.cc:1034] running on GCE node
I20260928 16:37:15.812923 16538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:15.814786 16538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:15.816056 16538 hybrid_clock.cc:648] HybridClock initialized: now 1790613435815991 us; error 60 us; skew 500 ppm
I20260928 16:37:15.816519 16538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:37:15.819541 16538 webserver.cc:460] Webserver started at http://127.10.247.254:39357/ using document root <none> and password file <none>
I20260928 16:37:15.820449 16538 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:15.820585 16538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:15.820993 16538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:15.824831 16538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "4439e078dfa54dce870381a1ea2f6760"
format_stamp: "Formatted at 2026-09-28 16:37:15 on dist-test-slave-9fn6"
server_key: "814f1689f0620d55089f0bdedbca5d96"
server_key_iv: "b0dc7e3edadbc724ce34cc9d01adda0a"
server_key_version: "encryptionkey@0"
I20260928 16:37:15.825860 16538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "4439e078dfa54dce870381a1ea2f6760"
format_stamp: "Formatted at 2026-09-28 16:37:15 on dist-test-slave-9fn6"
server_key: "814f1689f0620d55089f0bdedbca5d96"
server_key_iv: "b0dc7e3edadbc724ce34cc9d01adda0a"
server_key_version: "encryptionkey@0"
I20260928 16:37:15.832515 16538 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260928 16:37:15.837034 16552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:15.838989 16538 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:37:15.839218 16538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "4439e078dfa54dce870381a1ea2f6760"
format_stamp: "Formatted at 2026-09-28 16:37:15 on dist-test-slave-9fn6"
server_key: "814f1689f0620d55089f0bdedbca5d96"
server_key_iv: "b0dc7e3edadbc724ce34cc9d01adda0a"
server_key_version: "encryptionkey@0"
I20260928 16:37:15.839529 16538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:15.867237 16538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:15.868506 16538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:15.868968 16538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:15.890307 16538 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:37773
I20260928 16:37:15.890349 16603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:37773 every 8 connection(s)
I20260928 16:37:15.892148 16538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:15.897720 16604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:15.899989 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 16538
I20260928 16:37:15.900245 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:15.901228 11231 external_mini_cluster.cc:1428] Setting key ab653ca3da48277f22b521f4f1e077bc
I20260928 16:37:15.912420 16604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760: Bootstrap starting.
I20260928 16:37:15.916308 16604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:15.917079 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:15.903935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.10.247.254:37773)
0928 16:37:15.905369 (+ 1434us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:15.905384 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:15.905447 (+ 63us) server_negotiation.cc:407] Connection header received
0928 16:37:15.907377 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:15.907409 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:15.908023 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:15.908459 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:15.909187 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:15.910680 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:15.912086 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:15.912896 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:15.913077 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:15.914356 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:15.914365 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:15.914384 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:15.915002 (+ 618us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:15.915107 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:15.915129 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:15.915243 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:15.915498 (+ 255us) server_negotiation.cc:299] Negotiation successful
0928 16:37:15.915847 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":544,"threads_started":1}
I20260928 16:37:15.917770 16604 log.cc:824] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:15.921438 16604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760: No bootstrap required, opened a new log
I20260928 16:37:15.921918 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:37:15.930398 16604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4439e078dfa54dce870381a1ea2f6760" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37773 } }
I20260928 16:37:15.931027 16604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:15.931150 16604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4439e078dfa54dce870381a1ea2f6760, State: Initialized, Role: FOLLOWER
I20260928 16:37:15.931958 16604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4439e078dfa54dce870381a1ea2f6760" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37773 } }
I20260928 16:37:15.932292 16604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:15.932461 16604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:15.932701 16604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:15.936020 16604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4439e078dfa54dce870381a1ea2f6760" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37773 } }
I20260928 16:37:15.936905 16604 leader_election.cc:302] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4439e078dfa54dce870381a1ea2f6760; no voters:
I20260928 16:37:15.937753 16604 leader_election.cc:288] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:15.937942 16609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:15.938680 16609 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [term 1 LEADER]: Becoming Leader. State: Replica: 4439e078dfa54dce870381a1ea2f6760, State: Running, Role: LEADER
I20260928 16:37:15.939368 16609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4439e078dfa54dce870381a1ea2f6760" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37773 } }
I20260928 16:37:15.940244 16604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:15.942202 16610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4439e078dfa54dce870381a1ea2f6760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4439e078dfa54dce870381a1ea2f6760" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37773 } } }
I20260928 16:37:15.942554 16610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:15.943462 16615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:15.943246 16611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4439e078dfa54dce870381a1ea2f6760. Latest consensus state: current_term: 1 leader_uuid: "4439e078dfa54dce870381a1ea2f6760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4439e078dfa54dce870381a1ea2f6760" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37773 } } }
I20260928 16:37:15.943773 16611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:15.949678 16615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:15.962239 16615 catalog_manager.cc:1347] Generated new cluster ID: e0c6852ad30d411b9d7cfc5d497a4eb5
I20260928 16:37:15.962386 16615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:15.990682 16615 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:15.993083 16615 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:16.007210 16615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4439e078dfa54dce870381a1ea2f6760: Generated new TSK 0
I20260928 16:37:16.008687 16615 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 16:37:16.051612 16634 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260928 16:37:16.051764 16634 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 16:37:16.053575 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:16.031197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.10.247.254:37773)
0928 16:37:16.031446 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:16.031453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:16.031786 (+ 333us) server_negotiation.cc:407] Connection header received
0928 16:37:16.032079 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:16.032086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:16.032211 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:16.032440 (+ 229us) server_negotiation.cc:226] Negotiated authn=JWT
0928 16:37:16.032970 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.034202 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.035715 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.036439 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.036579 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:16.037206 (+ 627us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 16:37:16.052641 (+ 15435us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36523/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 16:37:16.053218 (+ 577us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33146: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36523/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":93}
W20260928 16:37:16.053642 16633 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:16.030971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37773 (local address 127.0.0.1:33146)
0928 16:37:16.031658 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:16.031678 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:16.031852 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:16.032433 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:16.032437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:16.032448 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0928 16:37:16.032777 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:16.032790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.034378 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.034383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:16.035559 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:16.035572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.036668 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.036676 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:16.036832 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:16.036875 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 16:37:16.052971 (+ 16096us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36523/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 16:37:16.053300 (+ 329us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.247.254:37773: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36523/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":368,"threads_started":1}
W20260928 16:37:16.053726 16607 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33146: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36523/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260928 16:37:16.054983 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 16538
I20260928 16:37:16.056463 16599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:16.170488 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 16538
2026-09-28T16:37:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (875 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260928 16:37:16.228665 11231 mini_oidc.cc:152] Starting JWKS server
I20260928 16:37:16.231011 11231 webserver.cc:460] Webserver started at https://127.0.0.1:38195/ using document root <none> and password file <none>
I20260928 16:37:16.231323 11231 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 16:37:16.232115 11231 webserver.cc:460] Webserver started at http://127.0.0.1:36537/ using document root <none> and password file <none>
2026-09-28T16:37:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:16Z Disabled control of system clock
I20260928 16:37:16.268455 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38563
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:38563
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38195/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 16:37:16.651000 16641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:16.651525 16641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:16.651705 16641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:16.659801 16641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:16.659955 16641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:16.659996 16641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:16.660058 16641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:16.660110 16641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 16:37:16.660177 16641 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 16:37:16.660218 16641 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38195/jwks.json
I20260928 16:37:16.670307 16641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:38563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38195/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:16.671993 16641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:16.674399 16641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:16.687633 16646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:16.687597 16647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:16.687673 16649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:16.688903 16641 server_base.cc:1034] running on GCE node
I20260928 16:37:16.689891 16641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:16.691680 16641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:16.692891 16641 hybrid_clock.cc:648] HybridClock initialized: now 1790613436692841 us; error 45 us; skew 500 ppm
I20260928 16:37:16.693281 16641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:37:16.695887 16641 webserver.cc:460] Webserver started at http://127.10.247.254:37683/ using document root <none> and password file <none>
I20260928 16:37:16.696759 16641 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:16.696877 16641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:16.697248 16641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:16.700716 16641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "2a8c8aac1fc9417ea3da59c8797bff0b"
format_stamp: "Formatted at 2026-09-28 16:37:16 on dist-test-slave-9fn6"
server_key: "afb6afd4d91a754f0a47931b10b01a4d"
server_key_iv: "90e2e1a8cb3ec9a4fe6b0259184cd25b"
server_key_version: "encryptionkey@0"
I20260928 16:37:16.701634 16641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "2a8c8aac1fc9417ea3da59c8797bff0b"
format_stamp: "Formatted at 2026-09-28 16:37:16 on dist-test-slave-9fn6"
server_key: "afb6afd4d91a754f0a47931b10b01a4d"
server_key_iv: "90e2e1a8cb3ec9a4fe6b0259184cd25b"
server_key_version: "encryptionkey@0"
I20260928 16:37:16.707734 16641 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260928 16:37:16.711899 16655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:16.713630 16641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:37:16.713817 16641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "2a8c8aac1fc9417ea3da59c8797bff0b"
format_stamp: "Formatted at 2026-09-28 16:37:16 on dist-test-slave-9fn6"
server_key: "afb6afd4d91a754f0a47931b10b01a4d"
server_key_iv: "90e2e1a8cb3ec9a4fe6b0259184cd25b"
server_key_version: "encryptionkey@0"
I20260928 16:37:16.714059 16641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:16.748993 16641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:16.750190 16641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:16.750550 16641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:16.771168 16641 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:38563
I20260928 16:37:16.771190 16706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:38563 every 8 connection(s)
I20260928 16:37:16.772864 16641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:16.775031 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 16641
I20260928 16:37:16.775246 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:16.775969 11231 external_mini_cluster.cc:1428] Setting key 859c85fef3305f65206db9313a9a3067
I20260928 16:37:16.777993 16707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:16.791285 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:16.778300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.10.247.254:38563)
0928 16:37:16.779427 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:16.779439 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:16.779489 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:37:16.781270 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:16.781296 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:16.781868 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:16.782252 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:16.782971 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.784435 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.786357 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.787062 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.787228 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:16.788530 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:16.788538 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:16.788555 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:16.789261 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:16.789350 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:16.789376 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:16.789479 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:16.789730 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 16:37:16.790142 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":414,"threads_started":1}
I20260928 16:37:16.792784 16707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b: Bootstrap starting.
I20260928 16:37:16.795986 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:37:16.796753 16707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:16.797955 16707 log.cc:824] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:16.801818 16707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b: No bootstrap required, opened a new log
I20260928 16:37:16.810709 16707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c8aac1fc9417ea3da59c8797bff0b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38563 } }
I20260928 16:37:16.811261 16707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:16.811373 16707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8c8aac1fc9417ea3da59c8797bff0b, State: Initialized, Role: FOLLOWER
I20260928 16:37:16.812135 16707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c8aac1fc9417ea3da59c8797bff0b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38563 } }
I20260928 16:37:16.812435 16707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:16.812582 16707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:16.812813 16707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:16.815454 16707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c8aac1fc9417ea3da59c8797bff0b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38563 } }
I20260928 16:37:16.816185 16707 leader_election.cc:302] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a8c8aac1fc9417ea3da59c8797bff0b; no voters:
I20260928 16:37:16.817188 16707 leader_election.cc:288] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:16.817358 16712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:16.818285 16712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [term 1 LEADER]: Becoming Leader. State: Replica: 2a8c8aac1fc9417ea3da59c8797bff0b, State: Running, Role: LEADER
I20260928 16:37:16.818933 16712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c8aac1fc9417ea3da59c8797bff0b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38563 } }
I20260928 16:37:16.819944 16707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:16.822324 16714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a8c8aac1fc9417ea3da59c8797bff0b. Latest consensus state: current_term: 1 leader_uuid: "2a8c8aac1fc9417ea3da59c8797bff0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c8aac1fc9417ea3da59c8797bff0b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38563 } } }
I20260928 16:37:16.822849 16714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:16.822363 16713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a8c8aac1fc9417ea3da59c8797bff0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c8aac1fc9417ea3da59c8797bff0b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38563 } } }
I20260928 16:37:16.823256 16713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:16.827986 16719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:16.833225 16719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:16.844132 16719 catalog_manager.cc:1347] Generated new cluster ID: 657da07f3b5344e281ede0529a7106fe
I20260928 16:37:16.844273 16719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:16.858996 16719 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:16.860687 16719 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:16.872519 16719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a8c8aac1fc9417ea3da59c8797bff0b: Generated new TSK 0
I20260928 16:37:16.873781 16719 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:37:16.909271 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:16.899297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.10.247.254:38563)
0928 16:37:16.899502 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:16.899508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:16.899911 (+ 403us) server_negotiation.cc:407] Connection header received
0928 16:37:16.900275 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:16.900280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:16.900397 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:16.900596 (+ 199us) server_negotiation.cc:226] Negotiated authn=JWT
0928 16:37:16.901231 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.902101 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.904261 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.904880 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.905040 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:16.905709 (+ 669us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 16:37:16.908453 (+ 2744us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 16:37:16.908881 (+ 428us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":52}
W20260928 16:37:16.909430 16710 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260928 16:37:16.911369 16736 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:16.899046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38563 (local address 127.0.0.1:57014)
0928 16:37:16.899778 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:16.899800 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:16.899991 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:16.900649 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:16.900655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:16.900670 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0928 16:37:16.901026 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:16.901040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.902262 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.902268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:16.904059 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:16.904078 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:16.905109 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:16.905116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:16.905325 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:16.905383 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 16:37:16.908761 (+ 3378us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 16:37:16.910878 (+ 2117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.247.254:38563: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":1692,"thread_start_us":449,"threads_started":1}
I20260928 16:37:16.912832 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 16641
I20260928 16:37:16.914384 16702 generic_service.cc:196] Checking for leaks (request via RPC)
E20260928 16:37:16.921244 16737 webserver.cc:533] Webserver: sslize failed:
E20260928 16:37:16.921366 16737 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 16:37:17.019387 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 16641
2026-09-28T16:37:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (847 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-28T16:37:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:17Z Disabled control of system clock
I20260928 16:37:17.110694 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37777
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37777
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/testchainca.cert with env {}
W20260928 16:37:17.492081 16742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:17.492496 16742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:17.492595 16742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:17.500916 16742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:17.501082 16742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:17.501123 16742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:17.501178 16742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:17.501232 16742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 16:37:17.501314 16742 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 16:37:17.501364 16742 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260928 16:37:17.512846 16742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37777
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:17.514613 16742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:17.516341 16742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:17.531216 16750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:17.531114 16747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:17.531183 16748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:17.534337 16742 server_base.cc:1034] running on GCE node
I20260928 16:37:17.535429 16742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:17.537348 16742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:17.538614 16742 hybrid_clock.cc:648] HybridClock initialized: now 1790613437538539 us; error 79 us; skew 500 ppm
I20260928 16:37:17.539187 16742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:37:17.542201 16742 webserver.cc:460] Webserver started at http://127.10.247.254:36029/ using document root <none> and password file <none>
I20260928 16:37:17.543079 16742 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:17.543210 16742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:17.543565 16742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:17.547351 16742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "6d662952e9fa4ea4963c2a60f7686f94"
format_stamp: "Formatted at 2026-09-28 16:37:17 on dist-test-slave-9fn6"
server_key: "ed211ad9e08d7016ff0013b22d6bcc4d"
server_key_iv: "18c39fa5c136925bc501ed18b2c96d64"
server_key_version: "encryptionkey@0"
I20260928 16:37:17.548377 16742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "6d662952e9fa4ea4963c2a60f7686f94"
format_stamp: "Formatted at 2026-09-28 16:37:17 on dist-test-slave-9fn6"
server_key: "ed211ad9e08d7016ff0013b22d6bcc4d"
server_key_iv: "18c39fa5c136925bc501ed18b2c96d64"
server_key_version: "encryptionkey@0"
I20260928 16:37:17.555023 16742 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260928 16:37:17.559425 16756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:17.561264 16742 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:17.561492 16742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "6d662952e9fa4ea4963c2a60f7686f94"
format_stamp: "Formatted at 2026-09-28 16:37:17 on dist-test-slave-9fn6"
server_key: "ed211ad9e08d7016ff0013b22d6bcc4d"
server_key_iv: "18c39fa5c136925bc501ed18b2c96d64"
server_key_version: "encryptionkey@0"
I20260928 16:37:17.561756 16742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:17.585899 16742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:17.587049 16742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:17.587409 16742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:17.608333 16742 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:37777
I20260928 16:37:17.608349 16807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:37777 every 8 connection(s)
I20260928 16:37:17.610110 16742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:17.615089 16808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:17.618705 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 16742
I20260928 16:37:17.618958 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:17.619874 11231 external_mini_cluster.cc:1428] Setting key c70b30f3caa75a3cd52a39980741e667
I20260928 16:37:17.630117 16808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94: Bootstrap starting.
I20260928 16:37:17.634384 16808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:17.635752 16808 log.cc:824] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:17.637063 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:17.622718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48452 (local address 127.10.247.254:37777)
0928 16:37:17.624072 (+ 1354us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:17.624114 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:17.624170 (+ 56us) server_negotiation.cc:407] Connection header received
0928 16:37:17.626117 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:17.626151 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:17.626818 (+ 667us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:17.627343 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:17.628115 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:17.629630 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:17.631178 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:17.631990 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:17.632221 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:17.633720 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:17.633730 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:17.633748 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:37:17.634534 (+ 786us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:17.634694 (+ 160us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:37:17.634723 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:17.634872 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:17.635123 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 16:37:17.635527 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":468,"threads_started":1}
I20260928 16:37:17.639678 16808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94: No bootstrap required, opened a new log
I20260928 16:37:17.642221 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:37:17.648548 16808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d662952e9fa4ea4963c2a60f7686f94" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37777 } }
I20260928 16:37:17.649119 16808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:17.649223 16808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d662952e9fa4ea4963c2a60f7686f94, State: Initialized, Role: FOLLOWER
I20260928 16:37:17.649964 16808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d662952e9fa4ea4963c2a60f7686f94" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37777 } }
I20260928 16:37:17.650243 16808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:17.650393 16808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:17.650583 16808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:17.653415 16808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d662952e9fa4ea4963c2a60f7686f94" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37777 } }
I20260928 16:37:17.654083 16808 leader_election.cc:302] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d662952e9fa4ea4963c2a60f7686f94; no voters:
I20260928 16:37:17.654805 16808 leader_election.cc:288] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:17.655045 16813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:17.655812 16813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [term 1 LEADER]: Becoming Leader. State: Replica: 6d662952e9fa4ea4963c2a60f7686f94, State: Running, Role: LEADER
I20260928 16:37:17.656461 16813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d662952e9fa4ea4963c2a60f7686f94" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37777 } }
I20260928 16:37:17.657295 16808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:17.659626 16815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d662952e9fa4ea4963c2a60f7686f94. Latest consensus state: current_term: 1 leader_uuid: "6d662952e9fa4ea4963c2a60f7686f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d662952e9fa4ea4963c2a60f7686f94" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37777 } } }
I20260928 16:37:17.660080 16815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:17.660773 16814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d662952e9fa4ea4963c2a60f7686f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d662952e9fa4ea4963c2a60f7686f94" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37777 } } }
I20260928 16:37:17.661151 16814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:17.663352 16819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:17.669706 16819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:17.680863 16819 catalog_manager.cc:1347] Generated new cluster ID: 6ff9cef3c7564bea87eb9fed254caf00
I20260928 16:37:17.681020 16819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:17.700374 16819 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:17.701970 16819 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:17.713995 16819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d662952e9fa4ea4963c2a60f7686f94: Generated new TSK 0
I20260928 16:37:17.715427 16819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:37:17.896471 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:17.731396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.10.247.254:37777)
0928 16:37:17.731655 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:17.731663 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:17.732181 (+ 518us) server_negotiation.cc:407] Connection header received
0928 16:37:17.732475 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:17.732482 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:17.732636 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:17.732859 (+ 223us) server_negotiation.cc:226] Negotiated authn=JWT
0928 16:37:17.733350 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:17.734526 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:17.736152 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:17.736951 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:17.737099 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:17.737828 (+ 729us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 16:37:17.895444 (+157616us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 16:37:17.896047 (+ 603us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48468: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":92}
W20260928 16:37:17.896688 16837 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:17.731196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37777 (local address 127.0.0.1:48468)
0928 16:37:17.732044 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:17.732064 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:17.732245 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:17.732878 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:17.732882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:17.732894 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0928 16:37:17.733172 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:17.733183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:17.734709 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:17.734714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:17.735950 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:17.735984 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:17.737217 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:17.737224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:17.737402 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:17.737459 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 16:37:17.895906 (+158447us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 16:37:17.896233 (+ 327us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.247.254:37777: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":486,"threads_started":1}
W20260928 16:37:17.896701 16811 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48468: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260928 16:37:17.898802 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 16742
I20260928 16:37:17.900322 16803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:18.023726 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 16742
2026-09-28T16:37:18Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (985 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (488 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (481 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:37:19 dist-test-slave-9fn6 krb5kdc[16852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:37:19 dist-test-slave-9fn6 krb5kdc[16852](info): set up 2 sockets
Sep 28 16:37:19 dist-test-slave-9fn6 krb5kdc[16852](info): commencing operation
krb5kdc: starting...
W20260928 16:37:20.025540 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:37:20 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:37:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:37:20.285988 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42085
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:42085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:20.667294 16865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:20.667711 16865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:20.667863 16865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:20.675746 16865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:20.675895 16865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:20.676002 16865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:20.676083 16865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:20.676147 16865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:20.687443 16865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:42085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:20.689399 16865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:20.691138 16865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:20.704955 16870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:20.705040 16873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:20.705003 16871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:20.705894 16865 server_base.cc:1034] running on GCE node
I20260928 16:37:20.706923 16865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:20.708844 16865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:20.710103 16865 hybrid_clock.cc:648] HybridClock initialized: now 1790613440710047 us; error 48 us; skew 500 ppm
I20260928 16:37:20.710516 16865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:20 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613440, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:20.714438 16865 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:37:20.716492 16865 webserver.cc:460] Webserver started at http://127.10.247.254:44123/ using document root <none> and password file <none>
I20260928 16:37:20.717424 16865 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:20.717564 16865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:20.717945 16865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:20.721849 16865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "b1aad44ef5ac473397e132ee1ee11ea4"
format_stamp: "Formatted at 2026-09-28 16:37:20 on dist-test-slave-9fn6"
server_key: "cb1d1019085cbab541cfe8c84a6a2f13"
server_key_iv: "483c44d5bb2687247d3e005bbcef6a7e"
server_key_version: "encryptionkey@0"
I20260928 16:37:20.723032 16865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "b1aad44ef5ac473397e132ee1ee11ea4"
format_stamp: "Formatted at 2026-09-28 16:37:20 on dist-test-slave-9fn6"
server_key: "cb1d1019085cbab541cfe8c84a6a2f13"
server_key_iv: "483c44d5bb2687247d3e005bbcef6a7e"
server_key_version: "encryptionkey@0"
I20260928 16:37:20.729976 16865 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260928 16:37:20.734679 16880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:20.736646 16865 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260928 16:37:20.736876 16865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "b1aad44ef5ac473397e132ee1ee11ea4"
format_stamp: "Formatted at 2026-09-28 16:37:20 on dist-test-slave-9fn6"
server_key: "cb1d1019085cbab541cfe8c84a6a2f13"
server_key_iv: "483c44d5bb2687247d3e005bbcef6a7e"
server_key_version: "encryptionkey@0"
I20260928 16:37:20.737135 16865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:20.779891 16865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:20.785513 16865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:20.785851 16865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:20.807389 16865 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:42085
I20260928 16:37:20.807405 16931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:42085 every 8 connection(s)
I20260928 16:37:20.809180 16865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:20.814026 16932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:20.814172 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 16865
I20260928 16:37:20.814414 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:20.815335 11231 external_mini_cluster.cc:1428] Setting key e1373a332276909f6be5c2e260400539
I20260928 16:37:20.831889 16932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4: Bootstrap starting.
Sep 28 16:37:20 dist-test-slave-9fn6 krb5kdc[16852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:20.836246 16932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:20.838032 16932 log.cc:824] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:20.842613 16932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4: No bootstrap required, opened a new log
I20260928 16:37:20.846487 16935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:20.818019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43934 (local address 127.10.247.254:42085)
0928 16:37:20.818925 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:20.818940 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:20.818997 (+ 57us) server_negotiation.cc:407] Connection header received
0928 16:37:20.822343 (+ 3346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:20.822389 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:20.823159 (+ 770us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:20.823909 (+ 750us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:20.825836 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:20.828074 (+ 2238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:20.829796 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:20.830978 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:20.831183 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:20.836191 (+ 5008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:20.836205 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:20.836236 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:20.836297 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:20.840567 (+ 4270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:20.841635 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:20.841670 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:20.841681 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:20.841962 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:20.842622 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:20.842627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:20.842630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:20.843425 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:20.843715 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:20.844340 (+ 625us) server_negotiation.cc:299] Negotiation successful
0928 16:37:20.845008 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":412,"threads_started":1}
I20260928 16:37:20.852016 16932 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aad44ef5ac473397e132ee1ee11ea4" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42085 } }
I20260928 16:37:20.852486 16932 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:20.852599 16932 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1aad44ef5ac473397e132ee1ee11ea4, State: Initialized, Role: FOLLOWER
I20260928 16:37:20.853292 16932 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aad44ef5ac473397e132ee1ee11ea4" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42085 } }
I20260928 16:37:20.853560 16932 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:20.853709 16932 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:20.853893 16932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:20.856498 16932 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aad44ef5ac473397e132ee1ee11ea4" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42085 } }
I20260928 16:37:20.857308 16932 leader_election.cc:302] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1aad44ef5ac473397e132ee1ee11ea4; no voters:
I20260928 16:37:20.858188 16932 leader_election.cc:288] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:20.858454 16937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:20.859241 16937 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [term 1 LEADER]: Becoming Leader. State: Replica: b1aad44ef5ac473397e132ee1ee11ea4, State: Running, Role: LEADER
I20260928 16:37:20.860040 16937 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aad44ef5ac473397e132ee1ee11ea4" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42085 } }
I20260928 16:37:20.861452 16932 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:20.863329 16938 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1aad44ef5ac473397e132ee1ee11ea4. Latest consensus state: current_term: 1 leader_uuid: "b1aad44ef5ac473397e132ee1ee11ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aad44ef5ac473397e132ee1ee11ea4" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42085 } } }
I20260928 16:37:20.863817 16938 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:20.865247 16939 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1aad44ef5ac473397e132ee1ee11ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aad44ef5ac473397e132ee1ee11ea4" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42085 } } }
I20260928 16:37:20.865703 16939 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:20.866344 16942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:20.877158 16942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:20.890350 16942 catalog_manager.cc:1347] Generated new cluster ID: 1116e183021a4078a6bb2307073d0d73
I20260928 16:37:20.890517 16942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:20.898877 16942 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:20.900586 16942 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:20.913137 16942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1aad44ef5ac473397e132ee1ee11ea4: Generated new TSK 0
I20260928 16:37:20.914367 16942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:37:20.985548 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42085
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:21.417215 16959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:21.417603 16959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:21.417761 16959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:21.425894 16959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:21.426043 16959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:21.426205 16959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:37:21.437497 16959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:21.439306 16959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:21.441027 16959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:21.456784 16965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:21.456797 16967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:21.457417 16964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:21.462476 16959 server_base.cc:1034] running on GCE node
I20260928 16:37:21.463383 16959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:21.464682 16959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:21.465921 16959 hybrid_clock.cc:648] HybridClock initialized: now 1790613441465857 us; error 54 us; skew 500 ppm
I20260928 16:37:21.466284 16959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:21 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613441, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:21.470032 16959 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:37:21.471882 16959 webserver.cc:460] Webserver started at http://127.10.247.193:45869/ using document root <none> and password file <none>
I20260928 16:37:21.472805 16959 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:21.472946 16959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:21.473291 16959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:21.476963 16959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "2bc765e85bde45a992abd33bc037850b"
format_stamp: "Formatted at 2026-09-28 16:37:21 on dist-test-slave-9fn6"
server_key: "cd737540662a705a400ab9753f5252f1"
server_key_iv: "a3fab282014c058c802139afb55d3556"
server_key_version: "encryptionkey@0"
I20260928 16:37:21.477928 16959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "2bc765e85bde45a992abd33bc037850b"
format_stamp: "Formatted at 2026-09-28 16:37:21 on dist-test-slave-9fn6"
server_key: "cd737540662a705a400ab9753f5252f1"
server_key_iv: "a3fab282014c058c802139afb55d3556"
server_key_version: "encryptionkey@0"
I20260928 16:37:21.484344 16959 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:37:21.488600 16974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:21.490193 16959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:37:21.490373 16959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "2bc765e85bde45a992abd33bc037850b"
format_stamp: "Formatted at 2026-09-28 16:37:21 on dist-test-slave-9fn6"
server_key: "cd737540662a705a400ab9753f5252f1"
server_key_iv: "a3fab282014c058c802139afb55d3556"
server_key_version: "encryptionkey@0"
I20260928 16:37:21.490615 16959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:21.511438 16959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:21.516817 16959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:21.517172 16959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:21.518384 16959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:21.520193 16959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:21.520282 16959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:21.520417 16959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:21.520474 16959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:21.556181 16959 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:34257
I20260928 16:37:21.556205 17086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:34257 every 8 connection(s)
I20260928 16:37:21.557818 16959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:21.560300 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 16959
I20260928 16:37:21.560539 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:37:21.561431 11231 external_mini_cluster.cc:1428] Setting key e7595f6a4c005a706a20935f157878db
Sep 28 16:37:21 dist-test-slave-9fn6 krb5kdc[16852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613441, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:21.582500 17089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:21.561308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:58651 (local address 127.10.247.254:42085)
0928 16:37:21.562024 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:21.562032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:21.563176 (+ 1144us) server_negotiation.cc:407] Connection header received
0928 16:37:21.565563 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:21.565570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:21.565676 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:21.565795 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:21.567879 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:21.568811 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:21.570659 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:21.571129 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:21.571271 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:21.575644 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:21.575650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:21.575654 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:21.575697 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:21.578816 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:21.579705 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:21.579710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:21.579713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:21.579879 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:21.580347 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:21.580352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:21.580354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:21.580751 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:21.580910 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:21.581787 (+ 877us) server_negotiation.cc:299] Negotiation successful
0928 16:37:21.582091 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":348,"threads_started":1}
I20260928 16:37:21.583861 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:21.562002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42085 (local address 127.10.247.193:58651)
0928 16:37:21.562994 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:21.563047 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:21.565222 (+ 2175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:21.566123 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:21.566135 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:21.566860 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:21.567650 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:21.567681 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:21.569009 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:21.569014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:21.570487 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:21.570503 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:21.571431 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:21.571441 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:21.571718 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:21.572546 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:21.572581 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:21.575352 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:21.578994 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:21.579005 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:21.579026 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:21.579567 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:21.580000 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:21.580005 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:21.580009 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:21.580227 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:21.580898 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:21.580907 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:21.581350 (+ 443us) client_negotiation.cc:769] Sending connection context
0928 16:37:21.581660 (+ 310us) client_negotiation.cc:240] Negotiation successful
0928 16:37:21.582372 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":369,"threads_started":1}
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
I20260928 16:37:21.585968 17087 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42085
I20260928 16:37:21.586333 17087 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:21.587265 17087 heartbeater.cc:511] Master 127.10.247.254:42085 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
I20260928 16:37:21.589901 16897 ts_manager.cc:194] Registered new tserver with Master: 2bc765e85bde45a992abd33bc037850b (127.10.247.193:34257)
I20260928 16:37:21.591653 16897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:58651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:37:21.621692 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42085
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:22.015215 17093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:22.015615 17093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:22.015753 17093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:22.024077 17093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:22.024253 17093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:22.024420 17093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:22.035648 17093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:22.037526 17093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:22.039168 17093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:22.055727 17099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:22.055764 17098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:22.055876 17101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:22.056680 17093 server_base.cc:1034] running on GCE node
I20260928 16:37:22.057632 17093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:22.058907 17093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:22.060175 17093 hybrid_clock.cc:648] HybridClock initialized: now 1790613442060109 us; error 55 us; skew 500 ppm
I20260928 16:37:22.060591 17093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:22.064720 17093 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:37:22.066780 17093 webserver.cc:460] Webserver started at http://127.10.247.194:44257/ using document root <none> and password file <none>
I20260928 16:37:22.067948 17093 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:22.068171 17093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:22.068581 17093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:22.072415 17093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "16bd95ef978b431e940063bb91c2c17a"
format_stamp: "Formatted at 2026-09-28 16:37:22 on dist-test-slave-9fn6"
server_key: "995e19fc86dab22c364a633c89249f80"
server_key_iv: "c800ca4bea593ac61f4c87677028aef2"
server_key_version: "encryptionkey@0"
I20260928 16:37:22.073446 17093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "16bd95ef978b431e940063bb91c2c17a"
format_stamp: "Formatted at 2026-09-28 16:37:22 on dist-test-slave-9fn6"
server_key: "995e19fc86dab22c364a633c89249f80"
server_key_iv: "c800ca4bea593ac61f4c87677028aef2"
server_key_version: "encryptionkey@0"
I20260928 16:37:22.079931 17093 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:37:22.084427 17108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:22.086345 17093 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:22.086534 17093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "16bd95ef978b431e940063bb91c2c17a"
format_stamp: "Formatted at 2026-09-28 16:37:22 on dist-test-slave-9fn6"
server_key: "995e19fc86dab22c364a633c89249f80"
server_key_iv: "c800ca4bea593ac61f4c87677028aef2"
server_key_version: "encryptionkey@0"
I20260928 16:37:22.086767 17093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:22.121364 17093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:22.126873 17093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:22.127216 17093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:22.128454 17093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:22.130203 17093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:22.130285 17093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:22.130422 17093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:22.130472 17093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:22.165891 17093 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:42469
I20260928 16:37:22.165931 17220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:42469 every 8 connection(s)
I20260928 16:37:22.167546 17093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:22.170683 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 17093
I20260928 16:37:22.170924 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:37:22.171635 11231 external_mini_cluster.cc:1428] Setting key b37433d6acf098061c604916a30eb5aa
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:22.195554 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:22.170931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:56339 (local address 127.10.247.254:42085)
0928 16:37:22.171534 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:22.171540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:22.173144 (+ 1604us) server_negotiation.cc:407] Connection header received
0928 16:37:22.175532 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:22.175538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:22.175643 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:22.175750 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:22.177927 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.178793 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.180820 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.181358 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.181496 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:22.186936 (+ 5440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:22.186957 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:22.186970 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:22.187046 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:22.190421 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.191947 (+ 1526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.191955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.191959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.192162 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.192903 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.192909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.192913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.193377 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:22.193530 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:22.194916 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0928 16:37:22.195215 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":306,"threads_started":1}
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
I20260928 16:37:22.197743 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:22.171837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42085 (local address 127.10.247.194:56339)
0928 16:37:22.172960 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:22.173008 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:22.175182 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:22.176091 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:22.176103 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:22.176885 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:22.177691 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:22.177716 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.178995 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.178999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:22.180595 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:22.180650 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.181680 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.181688 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:22.181983 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:22.182957 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:22.183013 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:22.186399 (+ 3386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:22.190685 (+ 4286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:22.190700 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:22.190727 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:22.191576 (+ 849us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:22.192308 (+ 732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:22.192314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:22.192318 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:22.192716 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:22.193551 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:22.193574 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:22.194305 (+ 731us) client_negotiation.cc:769] Sending connection context
0928 16:37:22.194762 (+ 457us) client_negotiation.cc:240] Negotiation successful
0928 16:37:22.195555 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":465,"threads_started":1}
I20260928 16:37:22.200091 17221 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42085
I20260928 16:37:22.200516 17221 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
I20260928 16:37:22.201711 17221 heartbeater.cc:511] Master 127.10.247.254:42085 requested a full tablet report, sending...
I20260928 16:37:22.203836 16897 ts_manager.cc:194] Registered new tserver with Master: 16bd95ef978b431e940063bb91c2c17a (127.10.247.194:42469)
I20260928 16:37:22.204916 16897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:56339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:37:22.233551 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42085
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:37:22.595872 17087 heartbeater.cc:503] Master 127.10.247.254:42085 was elected leader, sending a full tablet report...
W20260928 16:37:22.620059 17227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:22.620476 17227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:22.620647 17227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:22.629189 17227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:22.629345 17227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:22.629518 17227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:22.641009 17227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:22.642791 17227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:22.644462 17227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:22.660087 17233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:22.660172 17232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:22.660017 17235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:22.660957 17227 server_base.cc:1034] running on GCE node
I20260928 16:37:22.661710 17227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:22.662912 17227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:22.664179 17227 hybrid_clock.cc:648] HybridClock initialized: now 1790613442664123 us; error 38 us; skew 500 ppm
I20260928 16:37:22.664551 17227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:22.668449 17227 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:37:22.670245 17227 webserver.cc:460] Webserver started at http://127.10.247.195:36567/ using document root <none> and password file <none>
I20260928 16:37:22.671110 17227 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:22.671232 17227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:22.671602 17227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:22.675202 17227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "5d3a60eaa7ea4dccb5916ae9986231d6"
format_stamp: "Formatted at 2026-09-28 16:37:22 on dist-test-slave-9fn6"
server_key: "41c1a1a1b56709cb407bd20accf3530e"
server_key_iv: "8e30c356aadfcda3c61de6e6f7729f74"
server_key_version: "encryptionkey@0"
I20260928 16:37:22.676087 17227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "5d3a60eaa7ea4dccb5916ae9986231d6"
format_stamp: "Formatted at 2026-09-28 16:37:22 on dist-test-slave-9fn6"
server_key: "41c1a1a1b56709cb407bd20accf3530e"
server_key_iv: "8e30c356aadfcda3c61de6e6f7729f74"
server_key_version: "encryptionkey@0"
I20260928 16:37:22.682627 17227 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260928 16:37:22.686771 17242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:22.688367 17227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:37:22.688571 17227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "5d3a60eaa7ea4dccb5916ae9986231d6"
format_stamp: "Formatted at 2026-09-28 16:37:22 on dist-test-slave-9fn6"
server_key: "41c1a1a1b56709cb407bd20accf3530e"
server_key_iv: "8e30c356aadfcda3c61de6e6f7729f74"
server_key_version: "encryptionkey@0"
I20260928 16:37:22.688858 17227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:22.718899 17227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:22.724519 17227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:22.724912 17227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:22.726145 17227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:22.727876 17227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:22.727963 17227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:22.728072 17227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:22.728119 17227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:22.763882 17227 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:42455
I20260928 16:37:22.763932 17354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:42455 every 8 connection(s)
I20260928 16:37:22.765446 17227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:22.772462 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 17227
I20260928 16:37:22.772727 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:37:22.773406 11231 external_mini_cluster.cc:1428] Setting key 6beb8b8b9f4d23e16a51f820e6d97924
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:22.789388 17358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:22.768492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:54299 (local address 127.10.247.254:42085)
0928 16:37:22.769075 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:22.769081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:22.770226 (+ 1145us) server_negotiation.cc:407] Connection header received
0928 16:37:22.772566 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:22.772573 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:22.772716 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:22.772851 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:22.775139 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.776210 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.778081 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.778542 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.778669 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:22.783003 (+ 4334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:22.783009 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:22.783013 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:22.783062 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:22.785836 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.786677 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.786681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.786684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.786818 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.787290 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.787293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.787295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.787631 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:22.787779 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:22.788702 (+ 923us) server_negotiation.cc:299] Negotiation successful
0928 16:37:22.788969 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":314,"threads_started":1}
I20260928 16:37:22.790771 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:22.769116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42085 (local address 127.10.247.195:54299)
0928 16:37:22.769980 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:22.770036 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:22.772123 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:22.773205 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:22.773221 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:22.774135 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:22.774919 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:22.774952 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.776388 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.776392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:22.777905 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:22.777920 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.778816 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.778823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:22.779199 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:22.780018 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:22.780053 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:22.782715 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:22.786000 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:22.786011 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:22.786042 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:22.786533 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:22.786934 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:22.786939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:22.786943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:22.787169 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:22.787772 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:22.787782 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:22.788222 (+ 440us) client_negotiation.cc:769] Sending connection context
0928 16:37:22.788515 (+ 293us) client_negotiation.cc:240] Negotiation successful
0928 16:37:22.789237 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":325,"threads_started":1}
I20260928 16:37:22.792382 17355 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42085
I20260928 16:37:22.792845 17355 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:22.793843 17355 heartbeater.cc:511] Master 127.10.247.254:42085 requested a full tablet report, sending...
I20260928 16:37:22.795624 16897 ts_manager.cc:194] Registered new tserver with Master: 5d3a60eaa7ea4dccb5916ae9986231d6 (127.10.247.195:42455)
I20260928 16:37:22.796628 16897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:54299
I20260928 16:37:22.803102 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/test-admin.keytab.
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:22.835347 11231 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260928 16:37:22.842077 17364 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:22.838778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.10.247.193:34257)
0928 16:37:22.839525 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:22.839534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:22.839567 (+ 33us) server_negotiation.cc:407] Connection header received
0928 16:37:22.839649 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:22.839657 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:22.839934 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:22.840105 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:22.841578 (+ 1473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48874: BlockingRecv error: recv got EOF from 127.0.0.1:48874 (error 108)
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":379,"threads_started":1}
W20260928 16:37:22.842576 11231 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5cc' of type FILE
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:22.844471 11231 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:22.846213 11231 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:37:22.865073 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:22.848237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.10.247.193:34257)
0928 16:37:22.848473 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:22.848480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:22.848726 (+ 246us) server_negotiation.cc:407] Connection header received
0928 16:37:22.848921 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:22.848926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:22.849012 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:22.849105 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:22.850368 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.851525 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.852985 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.853995 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.854137 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:22.858776 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:22.858796 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:22.858828 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:22.858879 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:22.862038 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.862866 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.862875 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.862877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.863024 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.863383 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.863386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.863388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.863818 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:22.864062 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:22.864407 (+ 345us) server_negotiation.cc:299] Negotiation successful
0928 16:37:22.864734 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260928 16:37:22.872241 17364 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:22.870100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.10.247.193:34257)
0928 16:37:22.870284 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:22.870290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:22.870726 (+ 436us) server_negotiation.cc:407] Connection header received
0928 16:37:22.870899 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:22.870903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:22.870973 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:22.871065 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:22.871943 (+ 878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48898: BlockingRecv error: recv got EOF from 127.0.0.1:48898 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260928 16:37:22.872401 11231 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5cc' of type FILE
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:22.874135 11231 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:22.875460 11231 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:37:22.890821 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:22.877087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.10.247.193:34257)
0928 16:37:22.877317 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:22.877323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:22.877542 (+ 219us) server_negotiation.cc:407] Connection header received
0928 16:37:22.877722 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:22.877726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:22.877796 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:22.877884 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:22.878966 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.879761 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.881095 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.881564 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.881680 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:22.885486 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:22.885491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:22.885495 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:22.885534 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:22.888088 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.888775 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.888781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.888785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.888963 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.889355 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.889360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.889363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.889663 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:22.889930 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:22.890236 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 16:37:22.890463 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260928 16:37:22.896162 17364 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:22.894047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48908 (local address 127.10.247.193:34257)
0928 16:37:22.894202 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:22.894206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:22.894612 (+ 406us) server_negotiation.cc:407] Connection header received
0928 16:37:22.894806 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:22.894811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:22.894895 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:22.894981 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:22.895874 (+ 893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48908: BlockingRecv error: recv got EOF from 127.0.0.1:48908 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260928 16:37:22.896310 11231 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790613388455587-11231-0/krb5kdc/krb5cc' of type FILE
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:22.898133 11231 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:22.899609 11231 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 16:37:22 dist-test-slave-9fn6 krb5kdc[16852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:37:22.915885 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:22.901278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.10.247.193:34257)
0928 16:37:22.901520 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:22.901527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:22.901780 (+ 253us) server_negotiation.cc:407] Connection header received
0928 16:37:22.901966 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:22.901970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:22.902091 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:22.902183 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:22.903232 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.904036 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.905396 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.905832 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.905953 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:22.910053 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:22.910059 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:22.910063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:22.910109 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:22.913033 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.913766 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.913771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.913774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.913925 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.914304 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.914308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.914310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.914643 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:22.914977 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:22.915274 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 16:37:22.915516 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 16:37:22.917708 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 16959
I20260928 16:37:22.931073 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:22.918529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48940 (local address 127.10.247.193:34257)
0928 16:37:22.918690 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:22.918695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:22.919032 (+ 337us) server_negotiation.cc:407] Connection header received
0928 16:37:22.919229 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:22.919233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:22.919300 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:22.919386 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:22.920410 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.921227 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.922517 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:22.922927 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:22.923020 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:22.925459 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:22.925464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:22.925469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:22.925506 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:22.928249 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.929003 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.929008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.929011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.929192 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:22.929595 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:22.929599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:22.929602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:22.929943 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:22.930239 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:22.930524 (+ 285us) server_negotiation.cc:299] Negotiation successful
0928 16:37:22.930726 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 16:37:22.931849 17082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:23.028160 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 16959
I20260928 16:37:23.047533 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 17093
Sep 28 16:37:23 dist-test-slave-9fn6 krb5kdc[16852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:37:23.066218 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:23.048797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38376 (local address 127.10.247.194:42469)
0928 16:37:23.049475 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:23.049484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:23.049514 (+ 30us) server_negotiation.cc:407] Connection header received
0928 16:37:23.049598 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:23.049606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:23.049868 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:23.050078 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:23.051233 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:23.052485 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:23.053925 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:23.054772 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:23.054883 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:23.059204 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:23.059218 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:23.059238 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:23.059294 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:23.062606 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:23.063434 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:23.063443 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:23.063446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:23.063592 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:23.064085 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:23.064090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:23.064092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:23.064578 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:23.064943 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:23.065513 (+ 570us) server_negotiation.cc:299] Negotiation successful
0928 16:37:23.065787 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":359,"threads_started":1}
I20260928 16:37:23.067862 17216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:23.157982 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 17093
I20260928 16:37:23.175087 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 17227
Sep 28 16:37:23 dist-test-slave-9fn6 krb5kdc[16852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:37:23.194020 17379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:23.176252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.10.247.195:42455)
0928 16:37:23.176980 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:23.176988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:23.177024 (+ 36us) server_negotiation.cc:407] Connection header received
0928 16:37:23.177102 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:23.177110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:23.177397 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:23.177585 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:23.178855 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:23.180060 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:23.181609 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:23.182268 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:23.182400 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:23.187129 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:23.187145 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:23.187166 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:23.187221 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:23.190599 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:23.191439 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:23.191448 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:23.191450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:23.191616 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:23.191993 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:23.191998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:23.192002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:23.192443 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:23.192821 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:23.193276 (+ 455us) server_negotiation.cc:299] Negotiation successful
0928 16:37:23.193601 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":373,"threads_started":1}
I20260928 16:37:23.195703 17350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:23.286407 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 17227
I20260928 16:37:23.302675 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 16865
I20260928 16:37:23.304384 16927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:23.422487 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 16865
2026-09-28T16:37:23Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4433 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:37:23 dist-test-slave-9fn6 krb5kdc[17384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:37:23 dist-test-slave-9fn6 krb5kdc[17384](info): set up 2 sockets
Sep 28 16:37:23 dist-test-slave-9fn6 krb5kdc[17384](info): commencing operation
krb5kdc: starting...
W20260928 16:37:24.465574 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:37:24 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.254.keytab.
Entry for principal oryx/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.254.keytab.
I20260928 16:37:24.712700 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44723
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.254.keytab
--principal=oryx/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:44723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:25.092686 17397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:25.093080 17397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:25.093217 17397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:25.101321 17397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:25.101466 17397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:25.101526 17397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:25.101609 17397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:25.101670 17397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:25.113058 17397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:44723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.254.keytab
--principal=oryx/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:25.114918 17397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:25.116727 17397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:25.130353 17405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:25.130436 17402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:25.130431 17403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:25.131381 17397 server_base.cc:1034] running on GCE node
I20260928 16:37:25.132370 17397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:25.134136 17397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:25.135360 17397 hybrid_clock.cc:648] HybridClock initialized: now 1790613445135310 us; error 39 us; skew 500 ppm
I20260928 16:37:25.135772 17397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:25 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613445, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:25.145421 17397 init.cc:377] Logged in from keytab as oryx/127.10.247.254@KRBTEST.COM (short username oryx)
I20260928 16:37:25.147389 17397 webserver.cc:460] Webserver started at http://127.10.247.254:44325/ using document root <none> and password file <none>
I20260928 16:37:25.148268 17397 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:25.148401 17397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:25.148801 17397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:25.152487 17397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "989b1693d84c4f1497a967c16aa39701"
format_stamp: "Formatted at 2026-09-28 16:37:25 on dist-test-slave-9fn6"
server_key: "1a8321d80b112de55e43979f2f002059"
server_key_iv: "0ea44fb5b6643080624c45e726147693"
server_key_version: "encryptionkey@0"
I20260928 16:37:25.153508 17397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "989b1693d84c4f1497a967c16aa39701"
format_stamp: "Formatted at 2026-09-28 16:37:25 on dist-test-slave-9fn6"
server_key: "1a8321d80b112de55e43979f2f002059"
server_key_iv: "0ea44fb5b6643080624c45e726147693"
server_key_version: "encryptionkey@0"
I20260928 16:37:25.159955 17397 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:37:25.164006 17412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:25.165745 17397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:37:25.165937 17397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "989b1693d84c4f1497a967c16aa39701"
format_stamp: "Formatted at 2026-09-28 16:37:25 on dist-test-slave-9fn6"
server_key: "1a8321d80b112de55e43979f2f002059"
server_key_iv: "0ea44fb5b6643080624c45e726147693"
server_key_version: "encryptionkey@0"
I20260928 16:37:25.166168 17397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:25.207580 17397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:25.212992 17397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:25.213322 17397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:25.233325 17397 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:44723
I20260928 16:37:25.233347 17463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:44723 every 8 connection(s)
I20260928 16:37:25.235030 17397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:25.239642 17464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:25.240437 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 17397
I20260928 16:37:25.240706 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:25.241665 11231 external_mini_cluster.cc:1428] Setting key 30a90bf2213b07cf7469bdb5052a0a73
I20260928 16:37:25.256117 17464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: Bootstrap starting.
Sep 28 16:37:25 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:25.260812 17464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:25.262202 17464 log.cc:824] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:25.266362 17464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: No bootstrap required, opened a new log
I20260928 16:37:25.268477 17467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:25.244239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60884 (local address 127.10.247.254:44723)
0928 16:37:25.245144 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:25.245157 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:25.245213 (+ 56us) server_negotiation.cc:407] Connection header received
0928 16:37:25.247052 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:25.247081 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:25.247569 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:25.247969 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:25.250022 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:25.251528 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:25.253461 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:25.254331 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:25.254496 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:25.259897 (+ 5401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:25.259907 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:25.259925 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:25.259982 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:25.263642 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:25.264677 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:25.264685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:25.264694 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:25.264889 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:25.265360 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:25.265364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:25.265366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:25.265977 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:25.266244 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:25.266758 (+ 514us) server_negotiation.cc:299] Negotiation successful
0928 16:37:25.267250 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":417,"threads_started":1}
I20260928 16:37:25.275183 17464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } }
I20260928 16:37:25.275607 17464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:25.275710 17464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989b1693d84c4f1497a967c16aa39701, State: Initialized, Role: FOLLOWER
I20260928 16:37:25.276449 17464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } }
I20260928 16:37:25.276782 17464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:25.276928 17464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:25.277112 17464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:25.279502 17464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } }
I20260928 16:37:25.280055 17464 leader_election.cc:302] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 989b1693d84c4f1497a967c16aa39701; no voters:
I20260928 16:37:25.280715 17464 leader_election.cc:288] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:25.280885 17469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:25.281982 17469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 1 LEADER]: Becoming Leader. State: Replica: 989b1693d84c4f1497a967c16aa39701, State: Running, Role: LEADER
I20260928 16:37:25.282533 17469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } }
I20260928 16:37:25.283143 17464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:25.285676 17470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } }
I20260928 16:37:25.286761 17470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:25.285705 17471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 989b1693d84c4f1497a967c16aa39701. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } }
I20260928 16:37:25.287829 17471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:25.292280 17478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:25.297837 17478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:25.308054 17478 catalog_manager.cc:1347] Generated new cluster ID: 0526e8f1799a42149a3bbf27fb8d4fc2
I20260928 16:37:25.308218 17478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:25.323761 17478 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:25.325464 17478 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:25.337949 17478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: Generated new TSK 0
I20260928 16:37:25.339061 17478 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab.
Entry for principal oryx/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab.
I20260928 16:37:25.396024 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44723
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab
--principal=oryx/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:25.775533 17490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:25.775892 17490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:25.776029 17490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:25.783969 17490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:25.784114 17490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:25.784291 17490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:37:25.795372 17490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab
--principal=oryx/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:25.797217 17490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:25.798841 17490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:25.813656 17495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:25.813592 17498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:25.813620 17496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:25.814625 17490 server_base.cc:1034] running on GCE node
I20260928 16:37:25.815383 17490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:25.816496 17490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:25.817703 17490 hybrid_clock.cc:648] HybridClock initialized: now 1790613445817642 us; error 44 us; skew 500 ppm
I20260928 16:37:25.818066 17490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:25 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613445, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:25.821581 17490 init.cc:377] Logged in from keytab as oryx/127.10.247.193@KRBTEST.COM (short username oryx)
I20260928 16:37:25.823292 17490 webserver.cc:460] Webserver started at http://127.10.247.193:39289/ using document root <none> and password file <none>
I20260928 16:37:25.824143 17490 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:25.824257 17490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:25.824563 17490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:25.828094 17490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "e708b3522244403a966782fd941ddf23"
format_stamp: "Formatted at 2026-09-28 16:37:25 on dist-test-slave-9fn6"
server_key: "e775bf1760a12ceceb1f7561004ad4d5"
server_key_iv: "176d7d158e459cf7392f91b1ad697404"
server_key_version: "encryptionkey@0"
I20260928 16:37:25.828950 17490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "e708b3522244403a966782fd941ddf23"
format_stamp: "Formatted at 2026-09-28 16:37:25 on dist-test-slave-9fn6"
server_key: "e775bf1760a12ceceb1f7561004ad4d5"
server_key_iv: "176d7d158e459cf7392f91b1ad697404"
server_key_version: "encryptionkey@0"
I20260928 16:37:25.834817 17490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:37:25.839078 17505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:25.840794 17490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:37:25.840968 17490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "e708b3522244403a966782fd941ddf23"
format_stamp: "Formatted at 2026-09-28 16:37:25 on dist-test-slave-9fn6"
server_key: "e775bf1760a12ceceb1f7561004ad4d5"
server_key_iv: "176d7d158e459cf7392f91b1ad697404"
server_key_version: "encryptionkey@0"
I20260928 16:37:25.841233 17490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:25.858835 17490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:25.863605 17490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:25.863919 17490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:25.865046 17490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:25.866577 17490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:25.866667 17490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:25.866798 17490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:25.866852 17490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:25.901240 17490 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:42405
I20260928 16:37:25.901278 17618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:42405 every 8 connection(s)
I20260928 16:37:25.902761 17490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:25.903383 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 17490
I20260928 16:37:25.903626 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:37:25.904582 11231 external_mini_cluster.cc:1428] Setting key cd5f953d4a8b06c6c1355f4b2a60feff
Sep 28 16:37:25 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613445, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:25.927095 17621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:25.905916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:50487 (local address 127.10.247.254:44723)
0928 16:37:25.906451 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:25.906458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:25.907653 (+ 1195us) server_negotiation.cc:407] Connection header received
0928 16:37:25.909863 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:25.909869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:25.909977 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:25.910086 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:25.911985 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:25.912833 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:25.914589 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:25.915046 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:25.915168 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:25.919768 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:25.919775 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:25.919778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:25.919824 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:25.922986 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:25.924017 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:25.924023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:25.924026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:25.924196 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:25.924777 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:25.924784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:25.924786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:25.925177 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:25.925354 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:25.926303 (+ 949us) server_negotiation.cc:299] Negotiation successful
0928 16:37:25.926612 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":241,"threads_started":1}
I20260928 16:37:25.928659 17622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:25.906533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.193:50487)
0928 16:37:25.907407 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:25.907451 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:25.909533 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:25.910395 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:25.910413 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:25.911046 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:25.911772 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:25.911794 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:25.913004 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:25.913008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:25.914421 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:25.914436 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:25.915301 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:25.915309 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:25.915590 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:25.916374 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:25.916415 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:25.919472 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:25.923212 (+ 3740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:25.923222 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:25.923242 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:25.923856 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:25.924327 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:25.924334 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:25.924338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:25.924643 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:25.925326 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:25.925337 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:25.925829 (+ 492us) client_negotiation.cc:769] Sending connection context
0928 16:37:25.926147 (+ 318us) client_negotiation.cc:240] Negotiation successful
0928 16:37:25.926852 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":328,"threads_started":1}
WARNING: no policy specified for oryx/127.10.247.194@KRBTEST.COM; defaulting to no policy
I20260928 16:37:25.931008 17619 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44723
I20260928 16:37:25.931427 17619 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:25.932549 17619 heartbeater.cc:511] Master 127.10.247.254:44723 requested a full tablet report, sending...
I20260928 16:37:25.935518 17429 ts_manager.cc:194] Registered new tserver with Master: e708b3522244403a966782fd941ddf23 (127.10.247.193:42405)
I20260928 16:37:25.937776 17429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:50487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab.
Entry for principal oryx/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab.
I20260928 16:37:25.972122 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44723
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab
--principal=oryx/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:26.344857 17625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:26.345263 17625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:26.345419 17625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:26.353324 17625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:26.353477 17625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:26.353645 17625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:26.364436 17625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab
--principal=oryx/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:26.366178 17625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:26.367823 17625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:26.383054 17631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:26.383028 17630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:26.383097 17633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:26.383864 17625 server_base.cc:1034] running on GCE node
I20260928 16:37:26.384675 17625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:26.385825 17625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:26.387036 17625 hybrid_clock.cc:648] HybridClock initialized: now 1790613446386977 us; error 47 us; skew 500 ppm
I20260928 16:37:26.387393 17625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:26 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613446, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:26.391124 17625 init.cc:377] Logged in from keytab as oryx/127.10.247.194@KRBTEST.COM (short username oryx)
I20260928 16:37:26.392948 17625 webserver.cc:460] Webserver started at http://127.10.247.194:46809/ using document root <none> and password file <none>
I20260928 16:37:26.393793 17625 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:26.393922 17625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:26.394272 17625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:26.398056 17625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "76a12c7a96564b11a3daadc9a95fa1f6"
format_stamp: "Formatted at 2026-09-28 16:37:26 on dist-test-slave-9fn6"
server_key: "b646510c993cca4d8367790c70dd3c23"
server_key_iv: "8b44e59547bcaf3f2990d6521185b290"
server_key_version: "encryptionkey@0"
I20260928 16:37:26.398940 17625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "76a12c7a96564b11a3daadc9a95fa1f6"
format_stamp: "Formatted at 2026-09-28 16:37:26 on dist-test-slave-9fn6"
server_key: "b646510c993cca4d8367790c70dd3c23"
server_key_iv: "8b44e59547bcaf3f2990d6521185b290"
server_key_version: "encryptionkey@0"
I20260928 16:37:26.405037 17625 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260928 16:37:26.409119 17640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:26.410765 17625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 16:37:26.410923 17625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "76a12c7a96564b11a3daadc9a95fa1f6"
format_stamp: "Formatted at 2026-09-28 16:37:26 on dist-test-slave-9fn6"
server_key: "b646510c993cca4d8367790c70dd3c23"
server_key_iv: "8b44e59547bcaf3f2990d6521185b290"
server_key_version: "encryptionkey@0"
I20260928 16:37:26.411140 17625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:26.437383 17625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:26.442256 17625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:26.442564 17625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:26.443655 17625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:26.445315 17625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:26.445402 17625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:26.445536 17625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:26.445593 17625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:26.478996 17625 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:36775
I20260928 16:37:26.479035 17752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:36775 every 8 connection(s)
I20260928 16:37:26.480463 17625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:26.488317 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 17625
I20260928 16:37:26.488535 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:37:26.489131 11231 external_mini_cluster.cc:1428] Setting key 9c6c7b26b316e067a94d53265af71609
Sep 28 16:37:26 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613446, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:26.503386 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:26.483400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:51831 (local address 127.10.247.254:44723)
0928 16:37:26.483960 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:26.483966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:26.485132 (+ 1166us) server_negotiation.cc:407] Connection header received
0928 16:37:26.487108 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:26.487114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:26.487204 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:26.487335 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:26.489146 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:26.490186 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:26.492207 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:26.492728 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:26.492836 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:26.497203 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:26.497208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:26.497211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:26.497249 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:26.500047 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:26.500864 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:26.500868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:26.500871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:26.501023 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:26.501449 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:26.501452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:26.501454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:26.501760 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:26.501904 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:26.502699 (+ 795us) server_negotiation.cc:299] Negotiation successful
0928 16:37:26.502962 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":274,"threads_started":1}
I20260928 16:37:26.504583 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:26.484139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.194:51831)
0928 16:37:26.484951 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:26.485007 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:26.486826 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:26.487601 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:26.487610 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:26.488212 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:26.488966 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:26.488990 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:26.490347 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:26.490352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:26.492035 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:26.492054 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:26.492985 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:26.492993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:26.493317 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:26.494181 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:26.494216 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:26.496950 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:26.500196 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:26.500207 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:26.500227 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:26.500744 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:26.501127 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:26.501132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:26.501136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:26.501338 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:26.501890 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:26.501899 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:26.502291 (+ 392us) client_negotiation.cc:769] Sending connection context
0928 16:37:26.502566 (+ 275us) client_negotiation.cc:240] Negotiation successful
0928 16:37:26.503199 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":308,"threads_started":1}
I20260928 16:37:26.506238 17753 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44723
I20260928 16:37:26.506578 17753 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:26.507404 17753 heartbeater.cc:511] Master 127.10.247.254:44723 requested a full tablet report, sending...
I20260928 16:37:26.509076 17429 ts_manager.cc:194] Registered new tserver with Master: 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775)
I20260928 16:37:26.509981 17429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:51831
WARNING: no policy specified for oryx/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab.
Entry for principal oryx/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab.
I20260928 16:37:26.544756 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44723
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab
--principal=oryx/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:26.921628 17759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:26.921964 17759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:26.922097 17759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:26.929960 17759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:26.930084 17759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:26.930236 17759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:26.940932 17759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab
--principal=oryx/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:26.942454 17619 heartbeater.cc:503] Master 127.10.247.254:44723 was elected leader, sending a full tablet report...
I20260928 16:37:26.942619 17759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:26.944361 17759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:26.959133 17765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:26.959172 17767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:26.959084 17764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:26.960011 17759 server_base.cc:1034] running on GCE node
I20260928 16:37:26.960685 17759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:26.961689 17759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:26.962914 17759 hybrid_clock.cc:648] HybridClock initialized: now 1790613446962849 us; error 48 us; skew 500 ppm
I20260928 16:37:26.963259 17759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:26 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613446, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:26.966714 17759 init.cc:377] Logged in from keytab as oryx/127.10.247.195@KRBTEST.COM (short username oryx)
I20260928 16:37:26.968297 17759 webserver.cc:460] Webserver started at http://127.10.247.195:38073/ using document root <none> and password file <none>
I20260928 16:37:26.969115 17759 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:26.969231 17759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:26.969627 17759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:26.973284 17759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "9c6b9625bcac486191f88591d1094bf6"
format_stamp: "Formatted at 2026-09-28 16:37:26 on dist-test-slave-9fn6"
server_key: "ec606c9965ae78823ae2c2bdcae2b581"
server_key_iv: "0153d91d577fedf925a030d26a9f3049"
server_key_version: "encryptionkey@0"
I20260928 16:37:26.974140 17759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "9c6b9625bcac486191f88591d1094bf6"
format_stamp: "Formatted at 2026-09-28 16:37:26 on dist-test-slave-9fn6"
server_key: "ec606c9965ae78823ae2c2bdcae2b581"
server_key_iv: "0153d91d577fedf925a030d26a9f3049"
server_key_version: "encryptionkey@0"
I20260928 16:37:26.979986 17759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 16:37:26.984223 17774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:26.985917 17759 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:26.986102 17759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "9c6b9625bcac486191f88591d1094bf6"
format_stamp: "Formatted at 2026-09-28 16:37:26 on dist-test-slave-9fn6"
server_key: "ec606c9965ae78823ae2c2bdcae2b581"
server_key_iv: "0153d91d577fedf925a030d26a9f3049"
server_key_version: "encryptionkey@0"
I20260928 16:37:26.986331 17759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:27.005728 17759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:27.010919 17759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:27.011238 17759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:27.012380 17759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:27.014052 17759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:27.014153 17759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:27.014277 17759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:27.014328 17759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:27.049207 17759 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:45135
I20260928 16:37:27.049226 17886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:45135 every 8 connection(s)
I20260928 16:37:27.050791 17759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:27.051622 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 17759
I20260928 16:37:27.051854 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:37:27.052516 11231 external_mini_cluster.cc:1428] Setting key c64a46b34f8452a810c8e897e0c89fab
Sep 28 16:37:27 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613446, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:27.073833 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.054128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:56073 (local address 127.10.247.254:44723)
0928 16:37:27.054661 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:27.054668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:27.056254 (+ 1586us) server_negotiation.cc:407] Connection header received
0928 16:37:27.058198 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:27.058203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:27.058307 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:27.058412 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:27.060385 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.061360 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.063053 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.063475 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.063601 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.067797 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:27.067803 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:27.067808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:27.067855 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:27.070807 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.071595 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.071600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.071602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.071751 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.072111 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.072114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.072116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.072417 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:27.072548 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:27.073306 (+ 758us) server_negotiation.cc:299] Negotiation successful
0928 16:37:27.073515 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":250,"threads_started":1}
I20260928 16:37:27.074995 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.054932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.195:56073)
0928 16:37:27.055829 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:27.055884 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:27.057906 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:27.058702 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:27.058713 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:27.059332 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:27.060181 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.060204 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.061530 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.061535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.062906 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.062919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.063716 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.063724 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.063973 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.064778 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:27.064806 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:27.067578 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:27.070940 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.070948 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.070965 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.071499 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.071833 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.071836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.071839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.072031 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.072532 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:27.072539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:27.072983 (+ 444us) client_negotiation.cc:769] Sending connection context
0928 16:37:27.073243 (+ 260us) client_negotiation.cc:240] Negotiation successful
0928 16:37:27.073712 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":316,"threads_started":1}
I20260928 16:37:27.076541 17887 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44723
I20260928 16:37:27.076920 17887 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:27.077806 17887 heartbeater.cc:511] Master 127.10.247.254:44723 requested a full tablet report, sending...
I20260928 16:37:27.079429 17429 ts_manager.cc:194] Registered new tserver with Master: 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135)
I20260928 16:37:27.080364 17429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:56073
I20260928 16:37:27.081557 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.247.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.253.keytab.
Entry for principal oryx/127.10.247.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.253.keytab.
I20260928 16:37:27.137112 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.253:42395
--webserver_interface=127.10.247.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.253.keytab
--principal=oryx/127.10.247.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:44723,127.10.247.253:42395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:37:27.513130 17753 heartbeater.cc:503] Master 127.10.247.254:44723 was elected leader, sending a full tablet report...
W20260928 16:37:27.517925 17893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:27.518302 17893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:27.518460 17893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:27.526500 17893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:27.526671 17893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:27.526759 17893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:27.526839 17893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:27.526902 17893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:27.538189 17893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:44723,127.10.247.253:42395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.253.keytab
--principal=oryx/127.10.247.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.253:42395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.247.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
W20260928 16:37:27.538661 17893 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.247.254:44723,127.10.247.253:42395'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260928 16:37:27.540047 17893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:27.541673 17893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:27.554877 17898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:27.554843 17901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:27.555061 17899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:27.555738 17893 server_base.cc:1034] running on GCE node
I20260928 16:37:27.556668 17893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:27.558305 17893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:27.559573 17893 hybrid_clock.cc:648] HybridClock initialized: now 1790613447559486 us; error 53 us; skew 500 ppm
I20260928 16:37:27.559943 17893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:27 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613447, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:27.563498 17893 init.cc:377] Logged in from keytab as oryx/127.10.247.253@KRBTEST.COM (short username oryx)
I20260928 16:37:27.565400 17893 webserver.cc:460] Webserver started at http://127.10.247.253:35197/ using document root <none> and password file <none>
I20260928 16:37:27.566258 17893 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:27.566382 17893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:27.566754 17893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:27.570444 17893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data/instance:
uuid: "41a40f5a3199491db32d561d5d2caad9"
format_stamp: "Formatted at 2026-09-28 16:37:27 on dist-test-slave-9fn6"
server_key: "3fbe070a16d9a1622a49fb5eae5c3540"
server_key_iv: "a8f1e0fa2625d0b2a831f0138ce783f8"
server_key_version: "encryptionkey@0"
I20260928 16:37:27.571533 17893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal/instance:
uuid: "41a40f5a3199491db32d561d5d2caad9"
format_stamp: "Formatted at 2026-09-28 16:37:27 on dist-test-slave-9fn6"
server_key: "3fbe070a16d9a1622a49fb5eae5c3540"
server_key_iv: "a8f1e0fa2625d0b2a831f0138ce783f8"
server_key_version: "encryptionkey@0"
I20260928 16:37:27.577924 17893 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260928 16:37:27.581923 17908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:27.583542 17893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260928 16:37:27.583724 17893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal
uuid: "41a40f5a3199491db32d561d5d2caad9"
format_stamp: "Formatted at 2026-09-28 16:37:27 on dist-test-slave-9fn6"
server_key: "3fbe070a16d9a1622a49fb5eae5c3540"
server_key_iv: "a8f1e0fa2625d0b2a831f0138ce783f8"
server_key_version: "encryptionkey@0"
I20260928 16:37:27.583958 17893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:27.599527 17893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:27.604425 17893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:27.604780 17893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:27.625433 17893 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.253:42395
I20260928 16:37:27.625455 17959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.253:42395 every 8 connection(s)
I20260928 16:37:27.627332 17893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
I20260928 16:37:27.632314 17960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:27.633908 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 17893
I20260928 16:37:27.643625 17960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:37:27 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.253@KRBTEST.COM
Sep 28 16:37:27 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613447, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.253@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:27.661907 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.636855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39870 (local address 127.10.247.253:42395)
0928 16:37:27.637805 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:27.637824 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:27.637879 (+ 55us) server_negotiation.cc:407] Connection header received
0928 16:37:27.639841 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:27.639873 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:27.640416 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:27.640879 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:27.642963 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.644431 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.646387 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.647221 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.647442 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.652529 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:27.652538 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:27.652549 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:27.652631 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:27.656529 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.657472 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.657482 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.657493 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.657701 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.658129 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.658133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.658135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.658844 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:27.659173 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:27.659660 (+ 487us) server_negotiation.cc:299] Negotiation successful
0928 16:37:27.660194 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":423,"threads_started":1}
I20260928 16:37:27.664523 17964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.647317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.10.247.254:44723)
0928 16:37:27.648182 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:27.648188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:27.649158 (+ 970us) server_negotiation.cc:407] Connection header received
0928 16:37:27.649364 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:27.649371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:27.649470 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:27.649604 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:27.650843 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.651734 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.653442 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.653925 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.654029 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.657970 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:27.657977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:27.657982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:27.658039 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:27.661480 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.662294 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.662300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.662303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.662470 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.662853 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.662858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.662861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.663347 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:27.663498 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:27.664045 (+ 547us) server_negotiation.cc:299] Negotiation successful
0928 16:37:27.664250 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":569,"threads_started":1}
I20260928 16:37:27.664784 17965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.647313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:60888)
0928 16:37:27.648993 (+ 1680us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:27.649041 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:27.649221 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:27.649676 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:27.649682 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:27.650375 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:27.650682 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.650691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.651895 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.651900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.653295 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.653309 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.654137 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.654148 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.654320 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.654892 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:27.654915 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:27.657764 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:27.661623 (+ 3859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.661632 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.661638 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.662135 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.662598 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.662601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.662603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.662741 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.663485 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:27.663492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:27.663708 (+ 216us) client_negotiation.cc:769] Sending connection context
0928 16:37:27.663917 (+ 209us) client_negotiation.cc:240] Negotiation successful
0928 16:37:27.664330 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1378,"thread_start_us":1193,"threads_started":1}
I20260928 16:37:27.666666 17960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:37:27 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613447, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.253@KRBTEST.COM for oryx/127.10.247.253@KRBTEST.COM
I20260928 16:37:27.679112 17965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.667474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.0.0.1:39882)
0928 16:37:27.667685 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:27.667702 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:27.667791 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:27.668054 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:27.668057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:27.668414 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:27.668567 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.668575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.669573 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.669577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.670532 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.670543 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.671188 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.671192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.671366 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.671918 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:27.671937 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:27.674178 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:27.676984 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.676987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.676990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.677429 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.677729 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.677732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.677734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.677851 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.678313 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:27.678316 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:27.678454 (+ 138us) client_negotiation.cc:769] Sending connection context
0928 16:37:27.678562 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 16:37:27.678786 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260928 16:37:27.679163 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.667568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.10.247.253:42395)
0928 16:37:27.667725 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:27.667730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:27.667745 (+ 15us) server_negotiation.cc:407] Connection header received
0928 16:37:27.667877 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:27.667881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:27.667937 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:27.668008 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:27.668720 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.669451 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.670649 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.671050 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.671153 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.674349 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:27.674353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:27.674357 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:27.674385 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:27.676866 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.677521 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.677525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.677527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.677652 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.677929 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.677932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.677934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.678222 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:27.678328 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:27.678654 (+ 326us) server_negotiation.cc:299] Negotiation successful
0928 16:37:27.678905 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":57}
I20260928 16:37:27.684870 17960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: Bootstrap starting.
I20260928 16:37:27.688707 17960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:27.689903 17960 log.cc:824] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:27.693346 17960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: No bootstrap required, opened a new log
I20260928 16:37:27.701193 17960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } }
I20260928 16:37:27.701567 17960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:27.701647 17960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a40f5a3199491db32d561d5d2caad9, State: Initialized, Role: FOLLOWER
I20260928 16:37:27.702242 17960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } }
I20260928 16:37:27.703164 17967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } } }
I20260928 16:37:27.703532 17967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:37:27.704289 17960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 16:37:27.714197 17978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:37:27.714382 17978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:37:27.716420 17967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:27.716820 17967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } }
I20260928 16:37:27.718338 17967 leader_election.cc:288] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 989b1693d84c4f1497a967c16aa39701 (127.10.247.254:44723)
I20260928 16:37:27.719049 17439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "41a40f5a3199491db32d561d5d2caad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "989b1693d84c4f1497a967c16aa39701" is_pre_election: true
I20260928 16:37:27.720351 17893 master_runner.cc:428] Detected that this master 41a40f5a3199491db32d561d5d2caad9 is joining an existing cluster
I20260928 16:37:27.720431 17893 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.247.253:42395
I20260928 16:37:27.720317 17912 leader_election.cc:302] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 41a40f5a3199491db32d561d5d2caad9; no voters: 989b1693d84c4f1497a967c16aa39701
I20260928 16:37:27.721082 17967 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260928 16:37:27.741354 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.725492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.10.247.253:42395)
0928 16:37:27.725745 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:27.725752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:27.725918 (+ 166us) server_negotiation.cc:407] Connection header received
0928 16:37:27.726166 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:27.726172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:27.726253 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:27.726336 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:27.728172 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.729488 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.731661 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.732433 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.732531 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.734875 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:27.734881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:27.734884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:27.734912 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:27.738225 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.739091 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.739096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.739100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.739253 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.739653 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.739657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.739661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.740023 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:27.740169 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:27.740687 (+ 518us) server_negotiation.cc:299] Negotiation successful
0928 16:37:27.740962 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 16:37:27.742553 17984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.725136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.0.0.1:39884)
0928 16:37:27.725812 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:27.725836 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:27.726037 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:27.726562 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:27.726567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:27.727507 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:27.728005 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.728020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.729745 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.729750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.731512 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.731527 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.732893 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.732898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.733111 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.733822 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:27.733840 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:27.734696 (+ 856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:27.738374 (+ 3678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.738378 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.738382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.738954 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.739377 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.739381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.739384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.739553 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.740239 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:27.740243 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:27.740393 (+ 150us) client_negotiation.cc:769] Sending connection context
0928 16:37:27.741773 (+ 1380us) client_negotiation.cc:240] Negotiation successful
0928 16:37:27.742084 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":626,"threads_started":1}
I20260928 16:37:27.751452 17964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.726418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.10.247.254:44723)
0928 16:37:27.726614 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:27.726620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:27.737215 (+ 10595us) server_negotiation.cc:407] Connection header received
0928 16:37:27.737501 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:27.737509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:27.737625 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:27.737708 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:27.739243 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.740364 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.742313 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.742984 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.743084 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.745588 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:27.745595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:27.745600 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:27.745641 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:27.748586 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.749350 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.749357 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.749361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.749506 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.749890 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.749896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.749900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.750233 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:27.750376 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:27.750843 (+ 467us) server_negotiation.cc:299] Negotiation successful
0928 16:37:27.751093 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 16:37:27.751658 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.726481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:60902)
0928 16:37:27.737091 (+ 10610us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:27.737114 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:27.737334 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:27.737940 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:27.737945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:27.738666 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:27.739071 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.739082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.740559 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.740565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.742146 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.742160 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.743404 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.743410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.743631 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.744342 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:27.744363 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:27.745390 (+ 1027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:27.748778 (+ 3388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.748783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.748786 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.749233 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.749610 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.749615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.749619 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.749779 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.750347 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:27.750352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:27.750582 (+ 230us) client_negotiation.cc:769] Sending connection context
0928 16:37:27.750699 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 16:37:27.751024 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10438,"thread_start_us":6598,"threads_started":1}
WARNING: no policy specified for oryx/127.10.247.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.252@KRBTEST.COM" created.
Sep 28 16:37:27 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613445, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.254@KRBTEST.COM for oryx/127.10.247.253@KRBTEST.COM
I20260928 16:37:27.775854 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.759188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39896 (local address 127.10.247.253:42395)
0928 16:37:27.760460 (+ 1272us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:27.760467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:27.760487 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:37:27.760601 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:27.760630 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:27.760723 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:27.760854 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:27.762311 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.763524 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.765124 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.765711 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.765831 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.769505 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:27.769514 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:27.769518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:27.769553 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:27.772678 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.773520 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.773526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.773530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.773685 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.774080 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.774085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.774089 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.774445 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:27.774592 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:27.775183 (+ 591us) server_negotiation.cc:299] Negotiation successful
0928 16:37:27.775429 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127}
I20260928 16:37:27.775908 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.759162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.0.0.1:39896)
0928 16:37:27.759917 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:27.759978 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:27.760168 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:27.760904 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:27.760914 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:27.761808 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:27.762134 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.762147 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.763675 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.763681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.764977 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.764991 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.765882 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.765888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.766106 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.766795 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:27.766815 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:27.769288 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:27.772829 (+ 3541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.772845 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.772855 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.773390 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.773798 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.773804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.773808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.773976 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.774589 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:27.774599 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:27.774801 (+ 202us) client_negotiation.cc:769] Sending connection context
0928 16:37:27.774998 (+ 197us) client_negotiation.cc:240] Negotiation successful
0928 16:37:27.775434 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":280,"threads_started":1}
I20260928 16:37:27.778371 17429 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.247.253:42395: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "41a40f5a3199491db32d561d5d2caad9"
member_type: NON_VOTER
last_known_addr {
host: "127.10.247.253"
port: 42395
}
attrs {
promote: true
}
}
dest_uuid: "989b1693d84c4f1497a967c16aa39701"
cas_config_opid_index: -1
I20260928 16:37:27.779731 17429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } }
I20260928 16:37:27.785861 17990 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 41a40f5a3199491db32d561d5d2caad9 (127.10.247.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } } }
I20260928 16:37:27.787911 17935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:27.792107 17935 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Refusing update from remote peer 989b1693d84c4f1497a967c16aa39701: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260928 16:37:27.791193 17990 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.247.253:42395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.252.keytab.
Entry for principal oryx/127.10.247.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.252.keytab.
I20260928 16:37:27.793372 17991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } } }
I20260928 16:37:27.793766 17991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:27.793845 17893 master.cc:537] Master@127.10.247.253:42395 shutting down...
I20260928 16:37:27.794776 17991 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:27.795779 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.252:38941
--webserver_interface=127.10.247.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.252.keytab
--principal=oryx/127.10.247.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:44723,127.10.247.253:42395,127.10.247.252:38941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:27.800729 17413 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260928 16:37:27.804785 17413 consensus_peers.cc:487] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 -> Peer 41a40f5a3199491db32d561d5d2caad9 (127.10.247.253:42395): Couldn't send request to peer 41a40f5a3199491db32d561d5d2caad9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 16:37:27.810634 17893 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:27.811293 17893 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:27.811553 17893 tablet_replica.cc:330] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: stopping tablet replica
I20260928 16:37:27.821168 17893 master.cc:552] Master@127.10.247.253:42395 shutdown complete.
I20260928 16:37:27.821408 17893 master_runner.cc:305] Clearing existing system tablet
I20260928 16:37:27.821995 17893 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:27.822109 17893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:27.829463 17893 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.007s
I20260928 16:37:27.831476 17996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:27.831900 17893 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:27.832233 17893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260928 16:37:27.832330 17893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal
uuid: "41a40f5a3199491db32d561d5d2caad9"
format_stamp: "Formatted at 2026-09-28 16:37:27 on dist-test-slave-9fn6"
server_key: "3fbe070a16d9a1622a49fb5eae5c3540"
server_key_iv: "a8f1e0fa2625d0b2a831f0138ce783f8"
server_key_version: "encryptionkey@0"
I20260928 16:37:27.838160 17893 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:27.845680 17893 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 16:37:27.845911 17893 log.cc:1196] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260928 16:37:27.846376 17893 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: Deleting consensus metadata
I20260928 16:37:27.846661 17893 master_runner.cc:315] Copying system tablet from 127.10.247.254:44723
I20260928 16:37:27.849661 17893 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.247.254:44723
I20260928 16:37:27.876677 17964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.850884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60918 (local address 127.10.247.254:44723)
0928 16:37:27.851104 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:27.851112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:27.860022 (+ 8910us) server_negotiation.cc:407] Connection header received
0928 16:37:27.860263 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:27.860270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:27.860404 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:27.860540 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:27.861933 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.863225 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.865268 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.865901 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.866036 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.869084 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:27.869092 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:27.869098 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:27.869150 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:27.873245 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.874144 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.874151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.874155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.874342 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:27.874818 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:27.874823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:27.874827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:27.875266 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:27.875432 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:27.875950 (+ 518us) server_negotiation.cc:299] Negotiation successful
0928 16:37:27.876238 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:37:27.876722 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:27.850735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:60918)
0928 16:37:27.859790 (+ 9055us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:27.859825 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:27.860093 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:27.860646 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:27.860651 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:27.861310 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:27.861724 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.861739 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.863397 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.863407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.865082 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:27.865102 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:27.866106 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:27.866113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:27.866330 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:27.867241 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:27.867271 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:27.868816 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:27.873408 (+ 4592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.873414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.873418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.873994 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.874466 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:27.874471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:27.874475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:27.874699 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:27.875397 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:27.875402 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:27.875620 (+ 218us) client_negotiation.cc:769] Sending connection context
0928 16:37:27.875808 (+ 188us) client_negotiation.cc:240] Negotiation successful
0928 16:37:27.876173 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8865,"thread_start_us":8650,"threads_started":1}
I20260928 16:37:27.878094 17449 tablet_copy_service.cc:140] P 989b1693d84c4f1497a967c16aa39701: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 41a40f5a3199491db32d561d5d2caad9 ({username='oryx', principal='oryx/127.10.247.253@KRBTEST.COM'} at 127.0.0.1:60918)
I20260928 16:37:27.878408 17449 tablet_copy_service.cc:161] P 989b1693d84c4f1497a967c16aa39701: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 41a40f5a3199491db32d561d5d2caad9 at {username='oryx', principal='oryx/127.10.247.253@KRBTEST.COM'} at 127.0.0.1:60918: session id = 41a40f5a3199491db32d561d5d2caad9-00000000000000000000000000000000
I20260928 16:37:27.883580 17449 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 16:37:27.886766 17893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 16:37:27.894941 17893 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: tablet copy: Starting download of 0 data blocks...
I20260928 16:37:27.895360 17893 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: tablet copy: Starting download of 1 WAL segments...
I20260928 16:37:27.899894 17893 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 16:37:27.905407 17449 tablet_copy_service.cc:342] P 989b1693d84c4f1497a967c16aa39701: Request end of tablet copy session 41a40f5a3199491db32d561d5d2caad9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.247.253@KRBTEST.COM'} at 127.0.0.1:60918
I20260928 16:37:27.905700 17449 tablet_copy_service.cc:433] P 989b1693d84c4f1497a967c16aa39701: ending tablet copy session 41a40f5a3199491db32d561d5d2caad9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 41a40f5a3199491db32d561d5d2caad9
I20260928 16:37:27.920235 17893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:27.923988 18004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:27.924551 18003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:27.938510 17893 server_base.cc:1034] running on GCE node
W20260928 16:37:27.938892 18006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:27.939491 17893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:27.940269 17893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:27.941486 17893 hybrid_clock.cc:648] HybridClock initialized: now 1790613447941412 us; error 44 us; skew 500 ppm
I20260928 16:37:27.941756 17893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:27 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613447, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:27.944239 17893 init.cc:377] Logged in from keytab as oryx/127.10.247.253@KRBTEST.COM (short username oryx)
I20260928 16:37:27.950987 17893 webserver.cc:460] Webserver started at http://127.10.247.253:33091/ using document root <none> and password file <none>
I20260928 16:37:27.951428 17893 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:27.951514 17893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:27.954965 17893 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260928 16:37:27.957217 18013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:27.957849 17893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 16:37:27.957988 17893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal
uuid: "41a40f5a3199491db32d561d5d2caad9"
format_stamp: "Formatted at 2026-09-28 16:37:27 on dist-test-slave-9fn6"
server_key: "3fbe070a16d9a1622a49fb5eae5c3540"
server_key_iv: "a8f1e0fa2625d0b2a831f0138ce783f8"
server_key_version: "encryptionkey@0"
I20260928 16:37:27.958180 17893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:27.990386 17893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:27.994761 17893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:28.041507 17893 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.253:42395
I20260928 16:37:28.041558 18064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.253:42395 every 8 connection(s)
I20260928 16:37:28.042793 17893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
I20260928 16:37:28.050447 18065 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:37:28.054579 18065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: Bootstrap starting.
I20260928 16:37:28.070659 18065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:37:28.071237 18065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: Bootstrap complete.
I20260928 16:37:28.072335 18065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } }
I20260928 16:37:28.072593 18065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 41a40f5a3199491db32d561d5d2caad9, State: Initialized, Role: LEARNER
I20260928 16:37:28.072882 18065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } }
I20260928 16:37:28.073722 18068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } } }
I20260928 16:37:28.074105 18065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:28.074219 18068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: This master's current role is: LEARNER
I20260928 16:37:28.083577 17887 heartbeater.cc:503] Master 127.10.247.254:44723 was elected leader, sending a full tablet report...
I20260928 16:37:28.092931 18079 catalog_manager.cc:1259] Loaded cluster ID: 0526e8f1799a42149a3bbf27fb8d4fc2
I20260928 16:37:28.093052 18079 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: loading cluster ID for follower catalog manager: success
I20260928 16:37:28.096344 18079 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: acquiring CA information for follower catalog manager: success
I20260928 16:37:28.099502 18079 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:37:28.258149 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.245346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.0.0.1:39902)
0928 16:37:28.245756 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:28.245783 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:28.245946 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:28.252390 (+ 6444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:28.252397 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:28.252442 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:28.252706 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.252721 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.254305 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.254312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.255661 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.255675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.257223 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.257230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.257476 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.257555 (+ 79us) client_negotiation.cc:769] Sending connection context
0928 16:37:28.257656 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 16:37:28.257821 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260928 16:37:28.258870 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.246453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.10.247.253:42395)
0928 16:37:28.251907 (+ 5454us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.251914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.251942 (+ 28us) server_negotiation.cc:407] Connection header received
0928 16:37:28.252026 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.252031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.252136 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.252293 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:28.252877 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.254118 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.255835 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.256946 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.257285 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.257550 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.257682 (+ 132us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.257876 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5303,"thread_start_us":5034,"threads_started":1}
I20260928 16:37:28.260600 18040 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260928 16:37:28.261360 18068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 989b1693d84c4f1497a967c16aa39701. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } } }
I20260928 16:37:28.261672 18068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: This master's current role is: LEARNER
W20260928 16:37:28.334514 17994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:28.334884 17994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:28.335014 17994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:28.343665 17994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:28.343819 17994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:28.343888 17994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:28.343979 17994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:28.344043 17994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:28.356151 17994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:44723,127.10.247.253:42395,127.10.247.252:38941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.252.keytab
--principal=oryx/127.10.247.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.252:38941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.247.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:28.358460 17994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:28.360554 17994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:28.375609 18086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:28.376091 18088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:28.376744 18085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:28.377097 17994 server_base.cc:1034] running on GCE node
I20260928 16:37:28.378307 17994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:28.380247 17994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:28.381524 17994 hybrid_clock.cc:648] HybridClock initialized: now 1790613448381451 us; error 52 us; skew 500 ppm
I20260928 16:37:28.382020 17994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:28 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613448, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:28.386252 17994 init.cc:377] Logged in from keytab as oryx/127.10.247.252@KRBTEST.COM (short username oryx)
I20260928 16:37:28.388294 17994 webserver.cc:460] Webserver started at http://127.10.247.252:36535/ using document root <none> and password file <none>
I20260928 16:37:28.389199 17994 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:28.389319 17994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:28.389621 17994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:28.393460 17994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data/instance:
uuid: "510d281a5b5740d2ab8ddceacbd631d0"
format_stamp: "Formatted at 2026-09-28 16:37:28 on dist-test-slave-9fn6"
server_key: "aaa017c3e65ca7317d869375148c740a"
server_key_iv: "a4c85506f6691ec4ee0484c43a31cc75"
server_key_version: "encryptionkey@0"
I20260928 16:37:28.394354 17994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal/instance:
uuid: "510d281a5b5740d2ab8ddceacbd631d0"
format_stamp: "Formatted at 2026-09-28 16:37:28 on dist-test-slave-9fn6"
server_key: "aaa017c3e65ca7317d869375148c740a"
server_key_iv: "a4c85506f6691ec4ee0484c43a31cc75"
server_key_version: "encryptionkey@0"
I20260928 16:37:28.401108 17994 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260928 16:37:28.405265 18095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:28.406919 17994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 16:37:28.407092 17994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal
uuid: "510d281a5b5740d2ab8ddceacbd631d0"
format_stamp: "Formatted at 2026-09-28 16:37:28 on dist-test-slave-9fn6"
server_key: "aaa017c3e65ca7317d869375148c740a"
server_key_iv: "a4c85506f6691ec4ee0484c43a31cc75"
server_key_version: "encryptionkey@0"
I20260928 16:37:28.407300 17994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:28.441000 17994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:28.446163 17994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:28.446491 17994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:28.474392 17994 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.252:38941
I20260928 16:37:28.474393 18146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.252:38941 every 8 connection(s)
I20260928 16:37:28.476347 17994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
I20260928 16:37:28.482024 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 17994
I20260928 16:37:28.482379 18147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:28.495064 18147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:37:28 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.252@KRBTEST.COM
Sep 28 16:37:28 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613448, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.252@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:28.514389 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.485155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37698 (local address 127.10.247.252:38941)
0928 16:37:28.486133 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.486152 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.486224 (+ 72us) server_negotiation.cc:407] Connection header received
0928 16:37:28.488834 (+ 2610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.488873 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.489555 (+ 682us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.490131 (+ 576us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.491818 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.494421 (+ 2603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.495919 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.497141 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.497357 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.504007 (+ 6650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.504019 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.504036 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.504093 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.508776 (+ 4683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.509837 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.509845 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.509853 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.510088 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.510506 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.510512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.510515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.511337 (+ 822us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.511692 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.512174 (+ 482us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.512713 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":385,"threads_started":1}
I20260928 16:37:28.518327 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.498820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60924 (local address 127.10.247.254:44723)
0928 16:37:28.500323 (+ 1503us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.500328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.500344 (+ 16us) server_negotiation.cc:407] Connection header received
0928 16:37:28.500403 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.500407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.500484 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.500591 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.501935 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.502791 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.504971 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.505542 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.505653 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.510050 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.510057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.510062 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.510109 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.513870 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.515774 (+ 1904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.515780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.515783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.515997 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.516513 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.516519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.516522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.516965 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.517120 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.517657 (+ 537us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.517902 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1386,"thread_start_us":1236,"threads_started":1}
I20260928 16:37:28.518375 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.498668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:60924)
0928 16:37:28.499780 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:28.499826 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:28.500073 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:28.500710 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:28.500718 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:28.501484 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:28.501758 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.501765 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.502940 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.502944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.504812 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.504827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.505860 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.505868 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.506153 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.507014 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:28.507040 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:28.509855 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:28.514022 (+ 4167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.514032 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.514038 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.514526 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.516140 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.516146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.516149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.516356 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.517090 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:28.517098 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:28.517300 (+ 202us) client_negotiation.cc:769] Sending connection context
0928 16:37:28.517502 (+ 202us) client_negotiation.cc:240] Negotiation successful
0928 16:37:28.517902 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":471,"threads_started":1}
I20260928 16:37:28.519820 18147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:37:28 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613448, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.252@KRBTEST.COM for oryx/127.10.247.253@KRBTEST.COM
I20260928 16:37:28.534283 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.520850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.0.0.1:39918)
0928 16:37:28.521064 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:28.521085 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:28.521189 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:28.521584 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:28.521588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:28.521961 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:28.522200 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.522210 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.523322 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.523332 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.524681 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.524695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.525515 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.525519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.525660 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.526309 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:28.526329 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:28.528722 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:28.531864 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.531868 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.531872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.532315 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.532709 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.532712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.532715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.532852 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.533472 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:28.533475 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:28.533617 (+ 142us) client_negotiation.cc:769] Sending connection context
0928 16:37:28.533734 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 16:37:28.534002 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 16:37:28.534406 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.521089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39918 (local address 127.10.247.253:42395)
0928 16:37:28.521286 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.521292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.521308 (+ 16us) server_negotiation.cc:407] Connection header received
0928 16:37:28.521367 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.521371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.521447 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.521529 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.522342 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.523154 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.524859 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.525349 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.525465 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.528954 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.528960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.528963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.529007 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.531721 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.532414 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.532417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.532419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.532566 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.532960 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.532964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.532966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.533377 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.533496 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.533885 (+ 389us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.534101 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:37:28.535624 18147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:37:28 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613448, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.252@KRBTEST.COM for oryx/127.10.247.252@KRBTEST.COM
I20260928 16:37:28.549569 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.536637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37712 (local address 127.10.247.252:38941)
0928 16:37:28.536802 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.536808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.536823 (+ 15us) server_negotiation.cc:407] Connection header received
0928 16:37:28.536865 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.536870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.536925 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.537003 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.537748 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.538579 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.539868 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.540290 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.540384 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.543910 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.543915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.543919 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.543952 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.547012 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.547647 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.547651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.547653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.547776 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.548095 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.548098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.548100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.548448 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.548558 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.548960 (+ 402us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.549277 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":44}
I20260928 16:37:28.549574 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.536394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:38941 (local address 127.0.0.1:37712)
0928 16:37:28.536581 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:28.536597 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:28.536751 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:28.537072 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:28.537077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:28.537443 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:28.537592 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.537600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.538709 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.538714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.539750 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.539763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.540482 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.540487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.540877 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.541494 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:28.541512 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:28.543716 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:28.547152 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.547156 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.547159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.547550 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.547852 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.547855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.547857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.548014 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.548545 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:28.548549 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:28.548748 (+ 199us) client_negotiation.cc:769] Sending connection context
0928 16:37:28.548867 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 16:37:28.549146 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 16:37:28.555837 18147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Bootstrap starting.
I20260928 16:37:28.560719 18147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:28.562111 18147 log.cc:824] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:28.565738 18147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: No bootstrap required, opened a new log
I20260928 16:37:28.575091 18147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } }
I20260928 16:37:28.575565 18147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:28.575690 18147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 510d281a5b5740d2ab8ddceacbd631d0, State: Initialized, Role: FOLLOWER
I20260928 16:37:28.576344 18147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } }
I20260928 16:37:28.577371 18154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } } }
I20260928 16:37:28.577765 18154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:37:28.578645 18147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 16:37:28.590797 18165 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:37:28.591001 18165 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:37:28.595323 17994 master_runner.cc:428] Detected that this master 510d281a5b5740d2ab8ddceacbd631d0 is joining an existing cluster
I20260928 16:37:28.595472 17994 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.247.252:38941
I20260928 16:37:28.622841 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 17490
I20260928 16:37:28.624029 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.603833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.10.247.252:38941)
0928 16:37:28.604018 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.604024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.607138 (+ 3114us) server_negotiation.cc:407] Connection header received
0928 16:37:28.607325 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.607334 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.607399 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.607459 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.609313 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.610447 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.612533 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.613246 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.613360 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.616004 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.616010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.616014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.616053 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.620667 (+ 4614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.621686 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.621691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.621695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.621849 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.622227 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.622231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.622234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.622716 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.622848 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.623350 (+ 502us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.623626 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 16:37:28.625211 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.604002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:38941 (local address 127.0.0.1:37714)
0928 16:37:28.607047 (+ 3045us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:28.607065 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:28.607214 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:28.607624 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:28.607628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:28.608708 (+ 1080us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:28.609165 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.609176 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.610667 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.610675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.612384 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.612417 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.613422 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.613428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.613634 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.614500 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:28.614545 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:28.615807 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:28.621047 (+ 5240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.621051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.621055 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.621570 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.621986 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.621990 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.621993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.622132 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.622926 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:28.622930 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:28.623077 (+ 147us) client_negotiation.cc:769] Sending connection context
0928 16:37:28.624286 (+ 1209us) client_negotiation.cc:240] Negotiation successful
0928 16:37:28.624665 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2866,"thread_start_us":2578,"threads_started":1}
I20260928 16:37:28.625217 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.601144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60936 (local address 127.10.247.254:44723)
0928 16:37:28.601378 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.601385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.603696 (+ 2311us) server_negotiation.cc:407] Connection header received
0928 16:37:28.603959 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.603965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.604078 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.604160 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.607233 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.608346 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.610533 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.611169 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.611290 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.615422 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.615431 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.615436 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.615489 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.620521 (+ 5032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.621466 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.621474 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.621478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.621663 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.622125 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.622130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.622133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.622570 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.622721 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.623488 (+ 767us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.624682 (+ 1194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260928 16:37:28.626876 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.601900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:60936)
0928 16:37:28.603588 (+ 1688us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:28.603605 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:28.603795 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:28.604402 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:28.604407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:28.605748 (+ 1341us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:28.607048 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.607063 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.608558 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.608563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.610342 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.610356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.611736 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.611742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.611926 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.613711 (+ 1785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:28.613734 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:28.615120 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:28.620756 (+ 5636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.620761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.620764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.621310 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.621819 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.621824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.621827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.622008 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.622795 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:28.622799 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:28.623217 (+ 418us) client_negotiation.cc:769] Sending connection context
0928 16:37:28.625873 (+ 2656us) client_negotiation.cc:240] Negotiation successful
0928 16:37:28.626253 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1547,"thread_start_us":1117,"threads_started":1}
I20260928 16:37:28.628266 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.601450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.0.0.1:39926)
0928 16:37:28.602656 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:28.602678 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:28.602934 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:28.603602 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:28.603607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:28.605192 (+ 1585us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:28.606211 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.606223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.607809 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.607814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.610092 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.610108 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.611532 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.611543 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.611822 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.612954 (+ 1132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:28.612988 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:28.614701 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:28.619517 (+ 4816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.619523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.619527 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.620925 (+ 1398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.624785 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.624792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.624796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.625165 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.626798 (+ 1633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:28.626807 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:28.627228 (+ 421us) client_negotiation.cc:769] Sending connection context
0928 16:37:28.627421 (+ 193us) client_negotiation.cc:240] Negotiation successful
0928 16:37:28.627827 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"mutex_wait_us":1,"thread_start_us":768,"threads_started":1}
I20260928 16:37:28.628304 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.601143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.10.247.253:42395)
0928 16:37:28.601377 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.601384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.602797 (+ 1413us) server_negotiation.cc:407] Connection header received
0928 16:37:28.603124 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.603132 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.603240 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.603324 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.606380 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.607587 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.610283 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.611059 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.611176 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.615026 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.615035 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.615040 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.615093 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.619309 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.624200 (+ 4891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.624206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.624210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.624566 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.625324 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.625330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.625335 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.626477 (+ 1142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.626654 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.627606 (+ 952us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.627932 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
Sep 28 16:37:28 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613445, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.254@KRBTEST.COM for oryx/127.10.247.252@KRBTEST.COM
Sep 28 16:37:28 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.193@KRBTEST.COM
I20260928 16:37:28.653117 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.634027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:38941 (local address 127.0.0.1:37724)
0928 16:37:28.634523 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:28.634547 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:28.634700 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:28.635171 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:28.635178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:28.636189 (+ 1011us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:28.636471 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.636486 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.638586 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.638593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.639979 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.639996 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.641285 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.641292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.641486 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.642270 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:28.642296 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:28.644840 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:28.649604 (+ 4764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.649612 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.649617 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.650239 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.650793 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.650799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.650803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.651005 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.651745 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:28.651751 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:28.651977 (+ 226us) client_negotiation.cc:769] Sending connection context
0928 16:37:28.652158 (+ 181us) client_negotiation.cc:240] Negotiation successful
0928 16:37:28.652599 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304}
I20260928 16:37:28.653256 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.634068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.10.247.252:38941)
0928 16:37:28.634280 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.634285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.634639 (+ 354us) server_negotiation.cc:407] Connection header received
0928 16:37:28.634899 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.634906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.635020 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.635135 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.636695 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.638430 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.640149 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.641016 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.641216 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.645063 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.645071 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.645077 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.645123 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.649369 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.650408 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.650415 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.650418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.650616 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.651137 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.651143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.651147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.651590 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.651758 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.652302 (+ 544us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.652695 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:37:28.655189 17429 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.247.252:38941: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0"
member_type: NON_VOTER
last_known_addr {
host: "127.10.247.252"
port: 38941
}
attrs {
promote: true
}
}
dest_uuid: "989b1693d84c4f1497a967c16aa39701"
cas_config_opid_index: 5
I20260928 16:37:28.656381 17429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } }
I20260928 16:37:28.658243 17990 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 510d281a5b5740d2ab8ddceacbd631d0 (127.10.247.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.663407 17990 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.247.252:38941
I20260928 16:37:28.664974 17994 master.cc:537] Master@127.10.247.252:38941 shutting down...
I20260928 16:37:28.666214 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.624155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50344 (local address 127.10.247.193:42405)
0928 16:37:28.639567 (+ 15412us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.639579 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.639618 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:37:28.639709 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.639719 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.640181 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.640363 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.642463 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.644200 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.646278 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.647826 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.647977 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.654865 (+ 6888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.654890 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.654917 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.654981 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.660119 (+ 5138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.661521 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.661540 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.661543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.661773 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.662459 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.662465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.662469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.663047 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.663420 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.665044 (+ 1624us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.665604 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15099,"thread_start_us":16662,"threads_started":1}
I20260928 16:37:28.668028 17614 generic_service.cc:196] Checking for leaks (request via RPC)
W20260928 16:37:28.678624 17416 consensus_peers.cc:487] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 -> Peer 510d281a5b5740d2ab8ddceacbd631d0 (127.10.247.252:38941): Couldn't send request to peer 510d281a5b5740d2ab8ddceacbd631d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 16:37:28.680483 18178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.681301 18178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:28.681371 18040 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 LEARNER]: Refusing update from remote peer 989b1693d84c4f1497a967c16aa39701: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 16:37:28.682543 18175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260928 16:37:28.693414 18040 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 510d281a5b5740d2ab8ddceacbd631d0 (127.10.247.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.697752 18068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: NON_VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: true } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.698046 18068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: This master's current role is: LEARNER
I20260928 16:37:28.699167 18175 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: attempting to promote NON_VOTER 41a40f5a3199491db32d561d5d2caad9 to VOTER
I20260928 16:37:28.700116 18175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } }
I20260928 16:37:28.703166 18040 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 LEARNER]: Refusing update from remote peer 989b1693d84c4f1497a967c16aa39701: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260928 16:37:28.703560 17416 consensus_peers.cc:487] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 -> Peer 510d281a5b5740d2ab8ddceacbd631d0 (127.10.247.252:38941): Couldn't send request to peer 510d281a5b5740d2ab8ddceacbd631d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 16:37:28.703908 18175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260928 16:37:28.705904 18101 tablet.cc:2369] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:37:28.708356 17994 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:28.708941 17994 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:28.709204 17994 tablet_replica.cc:330] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: stopping tablet replica
I20260928 16:37:28.709868 18178 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 41a40f5a3199491db32d561d5d2caad9 (127.10.247.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.715996 17990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.715762 18179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.716271 17990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:28.716295 18179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:28.717069 18039 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 41a40f5a3199491db32d561d5d2caad9 (127.10.247.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.717824 17994 master.cc:552] Master@127.10.247.252:38941 shutdown complete.
I20260928 16:37:28.717908 17994 master_runner.cc:305] Clearing existing system tablet
I20260928 16:37:28.718273 17994 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:28.718379 17994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:28.720911 17994 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260928 16:37:28.722646 18183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:28.722856 18068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.723107 17994 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:28.723223 18068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:37:28.723353 17994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260928 16:37:28.723464 17994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal
uuid: "510d281a5b5740d2ab8ddceacbd631d0"
format_stamp: "Formatted at 2026-09-28 16:37:28 on dist-test-slave-9fn6"
server_key: "aaa017c3e65ca7317d869375148c740a"
server_key_iv: "a4c85506f6691ec4ee0484c43a31cc75"
server_key_version: "encryptionkey@0"
I20260928 16:37:28.727511 17994 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:28.732969 17994 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 16:37:28.733150 17994 log.cc:1196] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260928 16:37:28.733585 17994 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Deleting consensus metadata
I20260928 16:37:28.733829 17994 master_runner.cc:315] Copying system tablet from 127.10.247.254:44723
I20260928 16:37:28.736737 17994 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: tablet copy: Beginning tablet copy session from remote peer at address 127.10.247.254:44723
I20260928 16:37:28.759739 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.737634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.10.247.254:44723)
0928 16:37:28.737944 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:28.737953 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:28.741391 (+ 3438us) server_negotiation.cc:407] Connection header received
0928 16:37:28.741615 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:28.741622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:28.741726 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:28.741821 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:28.742768 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.743584 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.744966 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.745427 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.745550 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.752905 (+ 7355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:28.752911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:28.752914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:28.752970 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:28.756061 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.757196 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.757203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.757208 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.757363 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:28.757868 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:28.757874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:28.757877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:28.758288 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:28.758404 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:28.758961 (+ 557us) server_negotiation.cc:299] Negotiation successful
0928 16:37:28.759288 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260928 16:37:28.759789 18189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:28.737564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:60952)
0928 16:37:28.741241 (+ 3677us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:28.741261 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:28.741466 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:28.741898 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:28.741902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:28.742362 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:28.742590 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.742604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.743752 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.743758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.744795 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:28.744811 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:28.745582 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:28.745588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:28.745823 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:28.751455 (+ 5632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:28.751491 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:28.752565 (+ 1074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:28.756279 (+ 3714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.756286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.756291 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.757053 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.757503 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:28.757507 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:28.757511 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:28.757722 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:28.758479 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:28.758483 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:28.758674 (+ 191us) client_negotiation.cc:769] Sending connection context
0928 16:37:28.758796 (+ 122us) client_negotiation.cc:240] Negotiation successful
0928 16:37:28.759291 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3496,"thread_start_us":3368,"threads_started":1}
I20260928 16:37:28.760848 17449 tablet_copy_service.cc:140] P 989b1693d84c4f1497a967c16aa39701: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 510d281a5b5740d2ab8ddceacbd631d0 ({username='oryx', principal='oryx/127.10.247.252@KRBTEST.COM'} at 127.0.0.1:60952)
I20260928 16:37:28.761042 17449 tablet_copy_service.cc:161] P 989b1693d84c4f1497a967c16aa39701: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 510d281a5b5740d2ab8ddceacbd631d0 at {username='oryx', principal='oryx/127.10.247.252@KRBTEST.COM'} at 127.0.0.1:60952: session id = 510d281a5b5740d2ab8ddceacbd631d0-00000000000000000000000000000000
I20260928 16:37:28.764009 17449 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 16:37:28.766074 17994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 16:37:28.771940 17994 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: tablet copy: Starting download of 0 data blocks...
I20260928 16:37:28.772264 17994 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: tablet copy: Starting download of 1 WAL segments...
I20260928 16:37:28.774987 17994 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 16:37:28.779110 17449 tablet_copy_service.cc:342] P 989b1693d84c4f1497a967c16aa39701: Request end of tablet copy session 510d281a5b5740d2ab8ddceacbd631d0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.247.252@KRBTEST.COM'} at 127.0.0.1:60952
I20260928 16:37:28.779275 17449 tablet_copy_service.cc:433] P 989b1693d84c4f1497a967c16aa39701: ending tablet copy session 510d281a5b5740d2ab8ddceacbd631d0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 510d281a5b5740d2ab8ddceacbd631d0
I20260928 16:37:28.793160 17994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:28.796319 18190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:28.801084 18191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:28.808771 17994 server_base.cc:1034] running on GCE node
W20260928 16:37:28.809639 18193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:28.810166 17994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:28.810932 17994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:28.812242 17994 hybrid_clock.cc:648] HybridClock initialized: now 1790613448812199 us; error 40 us; skew 500 ppm
I20260928 16:37:28.812497 17994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:28 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613448, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:28.814594 17994 init.cc:377] Logged in from keytab as oryx/127.10.247.252@KRBTEST.COM (short username oryx)
I20260928 16:37:28.821041 17994 webserver.cc:460] Webserver started at http://127.10.247.252:34239/ using document root <none> and password file <none>
I20260928 16:37:28.821411 17994 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:28.821516 17994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:28.824306 17994 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260928 16:37:28.826597 18200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:28.827052 17994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260928 16:37:28.827180 17994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal
uuid: "510d281a5b5740d2ab8ddceacbd631d0"
format_stamp: "Formatted at 2026-09-28 16:37:28 on dist-test-slave-9fn6"
server_key: "aaa017c3e65ca7317d869375148c740a"
server_key_iv: "a4c85506f6691ec4ee0484c43a31cc75"
server_key_version: "encryptionkey@0"
I20260928 16:37:28.827370 17994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:28.840884 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 17490
I20260928 16:37:28.854151 17994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:28.858844 17994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:28.871320 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:42405
--local_ip_for_outbound_sockets=127.10.247.193
--tserver_master_addrs=127.10.247.254:44723
--webserver_port=39289
--webserver_interface=127.10.247.193
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab
--principal=oryx/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.247.254:44723,127.10.247.253:42395
--tserver_master_addrs=127.10.247.254:44723,127.10.247.253:42395,127.10.247.252:38941 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:37:28.933700 17994 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.252:38941
I20260928 16:37:28.933727 18253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.252:38941 every 8 connection(s)
I20260928 16:37:28.935202 17994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
I20260928 16:37:28.941455 18254 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:37:28.946211 18254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Bootstrap starting.
I20260928 16:37:28.968338 18254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:37:28.969049 18254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Bootstrap complete.
I20260928 16:37:28.970527 18254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } }
I20260928 16:37:28.970898 18254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 510d281a5b5740d2ab8ddceacbd631d0, State: Initialized, Role: LEARNER
I20260928 16:37:28.971225 18254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } }
I20260928 16:37:28.972698 18254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:28.972286 18257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:28.973452 18257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: This master's current role is: LEARNER
I20260928 16:37:28.985283 18268 catalog_manager.cc:1259] Loaded cluster ID: 0526e8f1799a42149a3bbf27fb8d4fc2
I20260928 16:37:28.985441 18268 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: loading cluster ID for follower catalog manager: success
I20260928 16:37:28.989667 18268 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: acquiring CA information for follower catalog manager: success
I20260928 16:37:28.992780 18268 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:37:29.316239 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:29.308428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:38941 (local address 127.0.0.1:37728)
0928 16:37:29.309089 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:29.309126 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:29.309316 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:29.309955 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:29.309960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:29.309990 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:29.310235 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:29.310247 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.312281 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.312286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:29.313605 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:29.313624 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.315524 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.315529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:29.315692 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:29.315725 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:37:29.315799 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 16:37:29.315949 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":321,"threads_started":1}
I20260928 16:37:29.316498 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:29.308585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.10.247.252:38941)
0928 16:37:29.309483 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:29.309490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:29.309517 (+ 27us) server_negotiation.cc:407] Connection header received
0928 16:37:29.309622 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:29.309628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:29.309751 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:29.309927 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:29.310416 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.312073 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.313799 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.315253 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.315498 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:29.315807 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:29.315877 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 16:37:29.316059 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":540,"threads_started":1}
I20260928 16:37:29.318030 18228 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260928 16:37:29.319067 18257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 989b1693d84c4f1497a967c16aa39701. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:29.319469 18257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: This master's current role is: LEARNER
W20260928 16:37:29.362319 18212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:29.362694 18212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:29.362819 18212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:29.371863 18212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:29.372001 18212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:29.372145 18212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:37:29.384928 18212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab
--principal=oryx/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:42405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=39289
--tserver_master_addrs=127.10.247.254:44723,127.10.247.253:42395,127.10.247.252:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:29.386765 18212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:29.388540 18212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:29.405501 18276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:29.405853 18277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:29.406087 18279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:29.407367 18212 server_base.cc:1034] running on GCE node
I20260928 16:37:29.408105 18212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:29.409363 18212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:29.410581 18212 hybrid_clock.cc:648] HybridClock initialized: now 1790613449410513 us; error 62 us; skew 500 ppm
I20260928 16:37:29.410987 18212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:29 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613449, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:29.414808 18212 init.cc:377] Logged in from keytab as oryx/127.10.247.193@KRBTEST.COM (short username oryx)
I20260928 16:37:29.416697 18212 webserver.cc:460] Webserver started at http://127.10.247.193:39289/ using document root <none> and password file <none>
I20260928 16:37:29.417753 18212 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:29.418009 18212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:29.426126 18212 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 16:37:29.429911 18286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:29.431710 18212 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:37:29.431888 18212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "e708b3522244403a966782fd941ddf23"
format_stamp: "Formatted at 2026-09-28 16:37:25 on dist-test-slave-9fn6"
server_key: "e775bf1760a12ceceb1f7561004ad4d5"
server_key_iv: "176d7d158e459cf7392f91b1ad697404"
server_key_version: "encryptionkey@0"
I20260928 16:37:29.433158 18212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:29.453819 18212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:29.459914 18212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:29.460254 18212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:29.461819 18212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:29.463531 18212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:29.463629 18212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:29.463763 18212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:29.463845 18212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:29.516108 18212 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:42405
I20260928 16:37:29.516256 18398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:42405 every 8 connection(s)
I20260928 16:37:29.518684 18212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:29.526103 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 18212
I20260928 16:37:29.526587 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 17625
Sep 28 16:37:29 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613449, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
Sep 28 16:37:29 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613449, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.252@KRBTEST.COM
Sep 28 16:37:29 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.194@KRBTEST.COM
Sep 28 16:37:29 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613449, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.253@KRBTEST.COM
I20260928 16:37:29.554900 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:29.522794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:38167 (local address 127.10.247.252:38941)
0928 16:37:29.523872 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:29.523879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:29.526145 (+ 2266us) server_negotiation.cc:407] Connection header received
0928 16:37:29.529762 (+ 3617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:29.529769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:29.529872 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:29.529974 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:29.533788 (+ 3814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.534989 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.538693 (+ 3704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.539382 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.539485 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:29.545342 (+ 5857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:29.545348 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:29.545353 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:29.545399 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:29.549092 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:29.550326 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:29.550334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:29.550339 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:29.550585 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:29.551288 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:29.551293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:29.551297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:29.551761 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:29.551937 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:29.554097 (+ 2160us) server_negotiation.cc:299] Negotiation successful
0928 16:37:29.554445 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870}
I20260928 16:37:29.557431 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:29.523590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:38941 (local address 127.10.247.193:38167)
0928 16:37:29.525993 (+ 2403us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:29.526032 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:29.529450 (+ 3418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:29.530514 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:29.530519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:29.532209 (+ 1690us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:29.533543 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:29.533574 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.535230 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.535235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:29.538443 (+ 3208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:29.538461 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.539821 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.539827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:29.539994 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:29.541012 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:29.541049 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:29.545034 (+ 3985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:29.549265 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:29.549278 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:29.549304 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:29.550135 (+ 831us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:29.550750 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:29.550755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:29.550758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:29.551124 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:29.552026 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:29.552037 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:29.552778 (+ 741us) client_negotiation.cc:769] Sending connection context
0928 16:37:29.553206 (+ 428us) client_negotiation.cc:240] Negotiation successful
0928 16:37:29.555559 (+ 2353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2185,"mutex_wait_us":406,"spinlock_wait_cycles":60032,"thread_start_us":1808,"threads_started":1}
I20260928 16:37:29.557662 18409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:29.527759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.10.247.194:36775)
0928 16:37:29.529178 (+ 1419us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:29.529187 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:29.529222 (+ 35us) server_negotiation.cc:407] Connection header received
0928 16:37:29.529312 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:29.529322 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:29.529643 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:29.531011 (+ 1368us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:29.531992 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.533711 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.535987 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.537203 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.546003 (+ 8800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:29.548387 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:29.548411 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:29.548436 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:29.548493 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:29.553152 (+ 4659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:29.554323 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:29.554334 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:29.554337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:29.554532 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:29.555091 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:29.555097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:29.555100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:29.555727 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:29.556172 (+ 445us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:29.556506 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 16:37:29.556939 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1212,"thread_start_us":1146,"threads_started":1}
I20260928 16:37:29.559041 18403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:29.522900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:56581 (local address 127.10.247.253:42395)
0928 16:37:29.523906 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:29.523913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:29.525420 (+ 1507us) server_negotiation.cc:407] Connection header received
0928 16:37:29.529995 (+ 4575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:29.530002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:29.530126 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:29.530221 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:29.533809 (+ 3588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.535281 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.538804 (+ 3523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.540049 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.540177 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:29.547969 (+ 7792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:29.547976 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:29.547980 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:29.548056 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:29.553164 (+ 5108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:29.554045 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:29.554050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:29.554054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:29.554224 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:29.554904 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:29.554909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:29.554911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:29.555812 (+ 901us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:29.555986 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:29.558144 (+ 2158us) server_negotiation.cc:299] Negotiation successful
0928 16:37:29.558450 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":613,"threads_started":1}
I20260928 16:37:29.559872 17748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:29.560211 18399 heartbeater.cc:347] Connected to a master server at 127.10.247.252:38941
I20260928 16:37:29.560513 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:29.523590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.10.247.193:56581)
0928 16:37:29.525195 (+ 1605us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:29.525263 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:29.529824 (+ 4561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:29.530780 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:29.530784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:29.532209 (+ 1425us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:29.533543 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:29.533573 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.535644 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.535650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:29.538591 (+ 2941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:29.538608 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.541351 (+ 2743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.541360 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:29.541620 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:29.542467 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:29.542495 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:29.547669 (+ 5174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:29.553340 (+ 5671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:29.553345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:29.553348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:29.553867 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:29.554578 (+ 711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:29.554584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:29.554588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:29.554767 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:29.556893 (+ 2126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:29.556898 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:29.557140 (+ 242us) client_negotiation.cc:769] Sending connection context
0928 16:37:29.557293 (+ 153us) client_negotiation.cc:240] Negotiation successful
0928 16:37:29.559880 (+ 2587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1186,"mutex_wait_us":370,"spinlock_wait_cycles":30336,"thread_start_us":696,"threads_started":1}
I20260928 16:37:29.560657 18399 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:29.561825 18399 heartbeater.cc:511] Master 127.10.247.252:38941 requested a full tablet report, sending...
I20260928 16:37:29.562238 18401 heartbeater.cc:347] Connected to a master server at 127.10.247.253:42395
I20260928 16:37:29.562418 18407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:29.527253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:57719 (local address 127.10.247.254:44723)
0928 16:37:29.529098 (+ 1845us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:29.529104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:29.529189 (+ 85us) server_negotiation.cc:407] Connection header received
0928 16:37:29.529902 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:29.529908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:29.529997 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:29.530165 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:29.533814 (+ 3649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.535071 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.538648 (+ 3577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.539369 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.540521 (+ 1152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:29.547486 (+ 6965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:29.547502 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:29.547508 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:29.547598 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:29.554900 (+ 7302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:29.557173 (+ 2273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:29.557179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:29.557183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:29.557384 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:29.559669 (+ 2285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:29.559675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:29.559679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:29.560336 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:29.560536 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:29.561624 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0928 16:37:29.561971 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1692,"thread_start_us":1510,"threads_started":1}
I20260928 16:37:29.562525 18401 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:29.563324 18406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:29.526127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.193:57719)
0928 16:37:29.526804 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:29.526821 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:29.529450 (+ 2629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:29.530507 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:29.530520 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:29.532209 (+ 1689us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:29.533547 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:29.533574 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.535250 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.535254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:29.538443 (+ 3189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:29.538461 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:29.539650 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:29.539657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:29.539994 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:29.541034 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:29.541061 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:29.546217 (+ 5156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:29.556071 (+ 9854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:29.556076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:29.556080 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:29.556754 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:29.559292 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:29.559297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:29.559300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:29.559524 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:29.560877 (+ 1353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:29.560882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:29.561207 (+ 325us) client_negotiation.cc:769] Sending connection context
0928 16:37:29.561448 (+ 241us) client_negotiation.cc:240] Negotiation successful
0928 16:37:29.561818 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":508,"spinlock_wait_cycles":16128,"thread_start_us":327,"threads_started":1}
I20260928 16:37:29.563575 18401 heartbeater.cc:511] Master 127.10.247.253:42395 requested a full tablet report, sending...
I20260928 16:37:29.564908 18400 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44723
I20260928 16:37:29.565068 18400 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:29.565423 18218 ts_manager.cc:194] Registered new tserver with Master: e708b3522244403a966782fd941ddf23 (127.10.247.193:42405)
I20260928 16:37:29.565627 18030 ts_manager.cc:194] Registered new tserver with Master: e708b3522244403a966782fd941ddf23 (127.10.247.193:42405)
I20260928 16:37:29.566154 18400 heartbeater.cc:511] Master 127.10.247.254:44723 requested a full tablet report, sending...
I20260928 16:37:29.567806 17429 ts_manager.cc:194] Re-registered known tserver with Master: e708b3522244403a966782fd941ddf23 (127.10.247.193:42405)
I20260928 16:37:29.569061 17429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:57719
I20260928 16:37:29.663612 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 17625
I20260928 16:37:29.681445 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:36775
--local_ip_for_outbound_sockets=127.10.247.194
--tserver_master_addrs=127.10.247.254:44723
--webserver_port=46809
--webserver_interface=127.10.247.194
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab
--principal=oryx/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.247.254:44723,127.10.247.253:42395
--tserver_master_addrs=127.10.247.254:44723,127.10.247.253:42395,127.10.247.252:38941 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:37:29.937052 18269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: NON_VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: true } } }
I20260928 16:37:29.937645 18269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:29.947893 18412 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: attempting to promote NON_VOTER 510d281a5b5740d2ab8ddceacbd631d0 to VOTER
I20260928 16:37:29.948820 18412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } }
I20260928 16:37:29.951912 18039 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Refusing update from remote peer 989b1693d84c4f1497a967c16aa39701: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260928 16:37:29.952023 18228 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 LEARNER]: Refusing update from remote peer 989b1693d84c4f1497a967c16aa39701: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260928 16:37:29.952886 18413 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260928 16:37:29.953487 18412 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260928 16:37:29.957300 18413 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 510d281a5b5740d2ab8ddceacbd631d0 (127.10.247.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:29.958602 18039 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 510d281a5b5740d2ab8ddceacbd631d0 (127.10.247.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:29.959373 18228 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 510d281a5b5740d2ab8ddceacbd631d0 (127.10.247.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:29.961860 18413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:29.962186 18413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:29.962067 18269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:29.962320 18269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:29.962546 18413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:29.962764 18413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:29.964414 18419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:29.965268 18419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:37:29.965611 18420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "989b1693d84c4f1497a967c16aa39701" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:29.966058 18420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 16:37:30.102802 18411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:30.103201 18411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:30.103348 18411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:30.111590 18411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:30.111752 18411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:30.111930 18411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:30.123404 18411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab
--principal=oryx/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:36775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=46809
--tserver_master_addrs=127.10.247.254:44723,127.10.247.253:42395,127.10.247.252:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:30.125384 18411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:30.127156 18411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:30.144346 18425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:30.144407 18428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:30.144384 18426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:30.145198 18411 server_base.cc:1034] running on GCE node
I20260928 16:37:30.145932 18411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:30.147135 18411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:30.148381 18411 hybrid_clock.cc:648] HybridClock initialized: now 1790613450148318 us; error 47 us; skew 500 ppm
I20260928 16:37:30.148828 18411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:30 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613450, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:30.157420 18411 init.cc:377] Logged in from keytab as oryx/127.10.247.194@KRBTEST.COM (short username oryx)
I20260928 16:37:30.159540 18411 webserver.cc:460] Webserver started at http://127.10.247.194:46809/ using document root <none> and password file <none>
I20260928 16:37:30.160519 18411 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:30.160715 18411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:30.167876 18411 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 16:37:30.171579 18435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:30.173686 18411 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:30.173946 18411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "76a12c7a96564b11a3daadc9a95fa1f6"
format_stamp: "Formatted at 2026-09-28 16:37:26 on dist-test-slave-9fn6"
server_key: "b646510c993cca4d8367790c70dd3c23"
server_key_iv: "8b44e59547bcaf3f2990d6521185b290"
server_key_version: "encryptionkey@0"
I20260928 16:37:30.175347 18411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:30.202080 18411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:30.207607 18411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:30.207931 18411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:30.209235 18411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:30.211033 18411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:30.211148 18411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:30.211277 18411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:30.211350 18411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:30.251091 18411 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:36775
I20260928 16:37:30.251164 18547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:36775 every 8 connection(s)
I20260928 16:37:30.253119 18411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:30.263038 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 18411
I20260928 16:37:30.263479 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 17759
Sep 28 16:37:30 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613450, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.253@KRBTEST.COM
Sep 28 16:37:30 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613450, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.252@KRBTEST.COM
Sep 28 16:37:30 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.195@KRBTEST.COM
I20260928 16:37:30.301470 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.257064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:52413 (local address 127.10.247.252:38941)
0928 16:37:30.264640 (+ 7576us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:30.264648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:30.278641 (+ 13993us) server_negotiation.cc:407] Connection header received
0928 16:37:30.278809 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:30.278816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:30.278930 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:30.279074 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:30.281037 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.282470 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.285515 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.286481 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.286619 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.292324 (+ 5705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:30.292335 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:30.292341 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:30.292400 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:30.296886 (+ 4486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.298038 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.298046 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.298050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.298277 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.299003 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.299009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.299012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.299409 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:30.299544 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:30.300675 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0928 16:37:30.301002 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7349,"thread_start_us":7138,"threads_started":1}
I20260928 16:37:30.302834 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.257205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:35263 (local address 127.10.247.253:42395)
0928 16:37:30.265150 (+ 7945us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:30.265159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:30.270922 (+ 5763us) server_negotiation.cc:407] Connection header received
0928 16:37:30.275379 (+ 4457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:30.275403 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:30.275555 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:30.275666 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:30.279439 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.280679 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.284192 (+ 3513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.284966 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.285089 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.291427 (+ 6338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:30.291439 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:30.291445 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:30.291508 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:30.296307 (+ 4799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.298004 (+ 1697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.298011 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.298015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.298256 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.300757 (+ 2501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.300763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.300767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.301248 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:30.301412 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:30.302115 (+ 703us) server_negotiation.cc:299] Negotiation successful
0928 16:37:30.302400 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7782,"thread_start_us":10082,"threads_started":1}
I20260928 16:37:30.304522 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.257970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.10.247.194:35263)
0928 16:37:30.270602 (+ 12632us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:30.270679 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:30.274811 (+ 4132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:30.276504 (+ 1693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:30.276522 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:30.277815 (+ 1293us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:30.279132 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.279169 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.281082 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.281090 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.283986 (+ 2896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.284004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.285540 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.285550 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.285967 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.286850 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:30.286901 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:30.290958 (+ 4057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:30.296676 (+ 5718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.296694 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.296727 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.297777 (+ 1050us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:30.298461 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.298467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.298472 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.300574 (+ 2102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:30.301576 (+ 1002us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:30.301581 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:30.301791 (+ 210us) client_negotiation.cc:769] Sending connection context
0928 16:37:30.303225 (+ 1434us) client_negotiation.cc:240] Negotiation successful
0928 16:37:30.303604 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12102,"mutex_wait_us":6,"spinlock_wait_cycles":138496,"thread_start_us":19540,"threads_started":1}
I20260928 16:37:30.304550 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.257969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:38941 (local address 127.10.247.194:52413)
0928 16:37:30.278498 (+ 20529us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:30.278523 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:30.278676 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:30.279378 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:30.279384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:30.280338 (+ 954us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:30.280800 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.280814 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.282816 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.282823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.285274 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.285294 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.287021 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.287028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.287257 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.287991 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:30.288016 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:30.292002 (+ 3986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:30.297105 (+ 5103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.297111 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.297116 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.297856 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:30.298442 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.298448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.298453 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.298866 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:30.299628 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:30.299637 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:30.300159 (+ 522us) client_negotiation.cc:769] Sending connection context
0928 16:37:30.301774 (+ 1615us) client_negotiation.cc:240] Negotiation successful
0928 16:37:30.302583 (+ 809us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20316,"mutex_wait_us":40,"thread_start_us":14485,"threads_started":1}
I20260928 16:37:30.306517 18548 heartbeater.cc:347] Connected to a master server at 127.10.247.252:38941
I20260928 16:37:30.306620 18550 heartbeater.cc:347] Connected to a master server at 127.10.247.253:42395
I20260928 16:37:30.306943 18550 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:30.306949 18548 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:30.308477 18548 heartbeater.cc:511] Master 127.10.247.252:38941 requested a full tablet report, sending...
I20260928 16:37:30.308514 18550 heartbeater.cc:511] Master 127.10.247.253:42395 requested a full tablet report, sending...
I20260928 16:37:30.310237 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.265620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.10.247.195:45135)
0928 16:37:30.267501 (+ 1881us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:30.267516 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:30.267562 (+ 46us) server_negotiation.cc:407] Connection header received
0928 16:37:30.267677 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:30.267690 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:30.268172 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:30.268490 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:30.271802 (+ 3312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.273700 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.282329 (+ 8629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.283926 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.284113 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.298845 (+ 14732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:30.298872 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:30.298915 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:30.298975 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:30.304558 (+ 5583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.305854 (+ 1296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.305874 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.305878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.306113 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.306725 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.306731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.306733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.307281 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:30.307758 (+ 477us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:30.309430 (+ 1672us) server_negotiation.cc:299] Negotiation successful
0928 16:37:30.309765 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1581,"thread_start_us":1239,"threads_started":1}
I20260928 16:37:30.310859 18030 ts_manager.cc:194] Registered new tserver with Master: 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775)
I20260928 16:37:30.311007 18218 ts_manager.cc:194] Registered new tserver with Master: 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775)
I20260928 16:37:30.313095 17882 generic_service.cc:196] Checking for leaks (request via RPC)
Sep 28 16:37:30 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613450, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:30.325536 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.274961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:51409 (local address 127.10.247.254:44723)
0928 16:37:30.300278 (+ 25317us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:30.300287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:30.300314 (+ 27us) server_negotiation.cc:407] Connection header received
0928 16:37:30.300912 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:30.300923 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:30.301060 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:30.301206 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:30.303951 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.305815 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.309094 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.309824 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.309978 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.317931 (+ 7953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:30.317938 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:30.317942 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:30.318005 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:30.321944 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.322868 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.322874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.322877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.323047 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.323698 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.323704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.323707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.324115 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:30.324275 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:30.324799 (+ 524us) server_negotiation.cc:299] Negotiation successful
0928 16:37:30.325088 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25130,"thread_start_us":25777,"threads_started":1}
I20260928 16:37:30.325554 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.274892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.194:51409)
0928 16:37:30.299405 (+ 24513us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:30.299434 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:30.299604 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:30.301464 (+ 1860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:30.301470 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:30.302837 (+ 1367us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:30.303688 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.303707 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.306459 (+ 2752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.306466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.308850 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.308871 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.312646 (+ 3775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.312663 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.312930 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.313716 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:30.313752 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:30.317581 (+ 3829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:30.322192 (+ 4611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.322198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.322202 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.322689 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:30.323261 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.323268 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.323271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.323521 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:30.324275 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:30.324280 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:30.324471 (+ 191us) client_negotiation.cc:769] Sending connection context
0928 16:37:30.324600 (+ 129us) client_negotiation.cc:240] Negotiation successful
0928 16:37:30.325055 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24307,"thread_start_us":23955,"threads_started":1}
I20260928 16:37:30.326593 18549 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44723
I20260928 16:37:30.326793 18549 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:30.327454 18549 heartbeater.cc:511] Master 127.10.247.254:44723 requested a full tablet report, sending...
I20260928 16:37:30.328914 17429 ts_manager.cc:194] Re-registered known tserver with Master: 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775)
I20260928 16:37:30.329879 17429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:51409
I20260928 16:37:30.405435 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 17759
I20260928 16:37:30.422523 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:45135
--local_ip_for_outbound_sockets=127.10.247.195
--tserver_master_addrs=127.10.247.254:44723
--webserver_port=38073
--webserver_interface=127.10.247.195
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab
--principal=oryx/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.247.254:44723,127.10.247.253:42395
--tserver_master_addrs=127.10.247.254:44723,127.10.247.253:42395,127.10.247.252:38941 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:37:30.572856 18400 heartbeater.cc:503] Master 127.10.247.254:44723 was elected leader, sending a full tablet report...
W20260928 16:37:30.811576 18561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:30.811983 18561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:30.812109 18561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:30.820137 18561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:30.820286 18561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:30.820425 18561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:30.831838 18561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab
--principal=oryx/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:45135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=38073
--tserver_master_addrs=127.10.247.254:44723,127.10.247.253:42395,127.10.247.252:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:30.833688 18561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:30.835326 18561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:30.851011 18566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:30.851012 18569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:30.851111 18567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:30.851650 18561 server_base.cc:1034] running on GCE node
I20260928 16:37:30.852594 18561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:30.853698 18561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:30.854954 18561 hybrid_clock.cc:648] HybridClock initialized: now 1790613450854887 us; error 49 us; skew 500 ppm
I20260928 16:37:30.855319 18561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:30 dist-test-slave-9fn6 krb5kdc[17384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613450, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:30.858913 18561 init.cc:377] Logged in from keytab as oryx/127.10.247.195@KRBTEST.COM (short username oryx)
I20260928 16:37:30.860831 18561 webserver.cc:460] Webserver started at http://127.10.247.195:38073/ using document root <none> and password file <none>
I20260928 16:37:30.861675 18561 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:30.861799 18561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:30.868686 18561 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.003s
I20260928 16:37:30.872206 18576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:30.874023 18561 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:30.874243 18561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "9c6b9625bcac486191f88591d1094bf6"
format_stamp: "Formatted at 2026-09-28 16:37:26 on dist-test-slave-9fn6"
server_key: "ec606c9965ae78823ae2c2bdcae2b581"
server_key_iv: "0153d91d577fedf925a030d26a9f3049"
server_key_version: "encryptionkey@0"
I20260928 16:37:30.875456 18561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:30.905376 18561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:30.910630 18561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:30.910940 18561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:30.912171 18561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:30.913846 18561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:30.913949 18561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:30.914049 18561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:30.914106 18561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:30.950174 18561 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:45135
I20260928 16:37:30.950310 18689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:45135 every 8 connection(s)
I20260928 16:37:30.951920 18561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:30.961357 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 18561
Sep 28 16:37:30 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613450, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.253@KRBTEST.COM
Sep 28 16:37:30 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613450, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.252@KRBTEST.COM
Sep 28 16:37:30 dist-test-slave-9fn6 krb5kdc[17384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613450, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:30.996589 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.955460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:49375 (local address 127.10.247.253:42395)
0928 16:37:30.956024 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:30.956032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:30.958035 (+ 2003us) server_negotiation.cc:407] Connection header received
0928 16:37:30.960342 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:30.960349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:30.960464 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:30.960562 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:30.964273 (+ 3711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.965702 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.969327 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.970039 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.970163 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.977106 (+ 6943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:30.977111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:30.977115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:30.977173 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:30.980682 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.991008 (+ 10326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.991017 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.991022 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.991351 (+ 329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.992069 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.992077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.992080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.992666 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:30.992832 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:30.995519 (+ 2687us) server_negotiation.cc:299] Negotiation successful
0928 16:37:30.995851 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":372,"threads_started":1}
I20260928 16:37:30.998044 18697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.956277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.10.247.195:49375)
0928 16:37:30.957900 (+ 1623us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:30.957918 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:30.959975 (+ 2057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:30.961276 (+ 1301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:30.961289 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:30.962824 (+ 1535us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:30.963985 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.964017 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.966485 (+ 2468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.966492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.969131 (+ 2639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.969149 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.970645 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.970654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.970985 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.972851 (+ 1866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:30.972888 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:30.976815 (+ 3927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:30.981563 (+ 4748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.981573 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.981599 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.990806 (+ 9207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:30.991505 (+ 699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.991511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.991515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.991904 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:30.992822 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:30.992834 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:30.993518 (+ 684us) client_negotiation.cc:769] Sending connection context
0928 16:37:30.995327 (+ 1809us) client_negotiation.cc:240] Negotiation successful
0928 16:37:30.996074 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1466,"mutex_wait_us":31,"spinlock_wait_cycles":134144,"thread_start_us":1150,"threads_started":1}
I20260928 16:37:31.002179 18698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.958204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.195:43567)
0928 16:37:30.972271 (+ 14067us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:30.972297 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:30.972448 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:30.973350 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:30.973355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:30.974138 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:30.979739 (+ 5601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.979766 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.981556 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.981561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.983767 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.983791 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.984704 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.984713 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.984940 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.985708 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:30.985734 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:30.989450 (+ 3716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:30.998124 (+ 8674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.998130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.998134 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.998996 (+ 862us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:30.999569 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:30.999588 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:30.999594 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:30.999785 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:31.000792 (+ 1007us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:31.000799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:31.001018 (+ 219us) client_negotiation.cc:769] Sending connection context
0928 16:37:31.001171 (+ 153us) client_negotiation.cc:240] Negotiation successful
0928 16:37:31.001647 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13910,"thread_start_us":13697,"threads_started":1}
I20260928 16:37:31.003619 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.958406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:43567 (local address 127.10.247.254:44723)
0928 16:37:30.971381 (+ 12975us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:30.971388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:30.972388 (+ 1000us) server_negotiation.cc:407] Connection header received
0928 16:37:30.972649 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:30.972656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:30.972759 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:30.972878 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:30.980040 (+ 7162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.981381 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.983939 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.984404 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.984529 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.989740 (+ 5211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:30.989747 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:30.989751 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:30.989800 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:30.997920 (+ 8120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.999195 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.999203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.999208 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:30.999426 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:30.999918 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:30.999924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:30.999928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.000465 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:31.000696 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.002817 (+ 2121us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.003157 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12815,"thread_start_us":12689,"threads_started":1}
I20260928 16:37:31.005561 18691 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44723
I20260928 16:37:31.005992 18691 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:31.007417 18691 heartbeater.cc:511] Master 127.10.247.254:44723 requested a full tablet report, sending...
I20260928 16:37:31.010243 17429 ts_manager.cc:194] Re-registered known tserver with Master: 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135)
I20260928 16:37:31.011962 17429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:43567
I20260928 16:37:31.012463 18692 heartbeater.cc:347] Connected to a master server at 127.10.247.253:42395
I20260928 16:37:31.012673 18692 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:31.013326 18692 heartbeater.cc:511] Master 127.10.247.253:42395 requested a full tablet report, sending...
I20260928 16:37:31.015115 18695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.955637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:42163 (local address 127.10.247.252:38941)
0928 16:37:30.956473 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:30.956480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:30.957645 (+ 1165us) server_negotiation.cc:407] Connection header received
0928 16:37:30.960364 (+ 2719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:30.960370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:30.960464 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:30.960562 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:30.969362 (+ 8800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.970466 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.973317 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.974019 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.974170 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.000867 (+ 26697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:31.000875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:31.000881 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:31.000939 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:31.006957 (+ 6018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.009989 (+ 3032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.009997 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.010003 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.010295 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.012710 (+ 2415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.012716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.012719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.013370 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:31.013631 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.014201 (+ 570us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.014450 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":573,"threads_started":1}
I20260928 16:37:31.016696 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.956277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:38941 (local address 127.10.247.195:42163)
0928 16:37:30.957407 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:30.957469 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:30.959941 (+ 2472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:30.966933 (+ 6992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:30.966937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:30.968134 (+ 1197us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:30.968460 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.968473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.970948 (+ 2475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.970953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.973156 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:30.973169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:30.981514 (+ 8345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:30.981522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:30.981813 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:30.984163 (+ 2350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:30.984195 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:30.995640 (+ 11445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:31.008158 (+ 12518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:31.008165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:31.008170 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:31.008883 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:31.010488 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:31.010494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:31.010499 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:31.010739 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:31.013714 (+ 2975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:31.013719 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:31.013898 (+ 179us) client_negotiation.cc:769] Sending connection context
0928 16:37:31.015355 (+ 1457us) client_negotiation.cc:240] Negotiation successful
0928 16:37:31.015865 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"spinlock_wait_cycles":20736,"thread_start_us":466,"threads_started":1}
I20260928 16:37:31.017280 18030 ts_manager.cc:194] Registered new tserver with Master: 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135)
I20260928 16:37:31.017805 18690 heartbeater.cc:347] Connected to a master server at 127.10.247.252:38941
I20260928 16:37:31.017979 18690 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:31.018555 18690 heartbeater.cc:511] Master 127.10.247.252:38941 requested a full tablet report, sending...
I20260928 16:37:31.020164 18218 ts_manager.cc:194] Registered new tserver with Master: 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135)
I20260928 16:37:31.033632 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.982701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.10.247.254:44723)
0928 16:37:31.004079 (+ 21378us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.004086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.004108 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:37:31.004178 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.004185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.004285 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.004404 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:31.018272 (+ 13868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.019521 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.021598 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.022235 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.022317 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.026185 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:31.026192 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:31.026197 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:31.026251 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:31.029933 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.030991 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.030998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.031002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.031171 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.031665 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.031672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.031676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.032217 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:31.032535 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.032988 (+ 453us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.033257 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21232,"thread_start_us":29021,"threads_started":1}
I20260928 16:37:31.034368 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.974819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.10.247.253:42395)
0928 16:37:30.997005 (+ 22186us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:30.997011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:30.997032 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:37:30.997099 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:30.997104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:30.997192 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:30.997301 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:31.018926 (+ 21625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.020118 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.022105 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.022834 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.022979 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.026313 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:31.026319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:31.026324 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:31.026365 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:31.030311 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.031392 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.031399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.031402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.031605 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.032004 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.032010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.032014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.032352 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:31.032708 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.033326 (+ 618us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.033666 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22007,"thread_start_us":36513,"threads_started":1}
I20260928 16:37:31.034449 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:30.992969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37734 (local address 127.10.247.252:38941)
0928 16:37:30.993926 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:30.993933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:30.998293 (+ 4360us) server_negotiation.cc:407] Connection header received
0928 16:37:30.999040 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:30.999047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:30.999182 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:30.999281 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:31.006119 (+ 6838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.007456 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.019598 (+ 12142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.020383 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.020485 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.025382 (+ 4897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:31.025397 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:31.025403 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:31.025454 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:31.029653 (+ 4199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.030909 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.030918 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.030922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.031269 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.031876 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.031882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.031885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.032432 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:31.032825 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.033869 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.034141 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":617,"threads_started":1}
I20260928 16:37:31.040657 17429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:37:31.049960 17429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:37:31.109279 18722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.075770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:45135 (local address 127.0.0.1:35602)
0928 16:37:31.097928 (+ 22158us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:31.097960 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:31.098144 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:31.100898 (+ 2754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:31.100904 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:31.100932 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:31.101395 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.101410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.103280 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.103287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.105705 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.105718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.108340 (+ 2622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.108346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.108485 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.108514 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 16:37:31.108599 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 16:37:31.108789 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22000,"thread_start_us":28464,"threads_started":1}
I20260928 16:37:31.109764 18724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.076292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.10.247.195:45135)
0928 16:37:31.093500 (+ 17208us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.093516 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.098324 (+ 4808us) server_negotiation.cc:407] Connection header received
0928 16:37:31.098482 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.098493 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.098898 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.099164 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:31.101828 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.103104 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.105864 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.107665 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.107857 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.108280 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.108726 (+ 446us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.108919 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16877,"thread_start_us":17274,"threads_started":1}
I20260928 16:37:31.113611 18723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.076054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:42405 (local address 127.0.0.1:50358)
0928 16:37:31.103807 (+ 27753us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:31.103829 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:31.103983 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:31.104971 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:31.104976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:31.105000 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:31.105503 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.105516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.107601 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.107606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.110322 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.110341 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.112696 (+ 2355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.112704 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.112907 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.112935 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 16:37:31.113022 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 16:37:31.113173 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27597,"mutex_wait_us":33,"thread_start_us":27398,"threads_started":1}
I20260928 16:37:31.113610 18726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.076895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.10.247.193:42405)
0928 16:37:31.097246 (+ 20351us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.097259 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.104008 (+ 6749us) server_negotiation.cc:407] Connection header received
0928 16:37:31.104215 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.104227 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.104683 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.105014 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:31.105776 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.107377 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.110540 (+ 3163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.112403 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.112580 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.112814 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.113106 (+ 292us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.113288 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20136,"thread_start_us":19949,"threads_started":1}
I20260928 16:37:31.116012 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.076121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:36775 (local address 127.0.0.1:57348)
0928 16:37:31.104565 (+ 28444us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:31.104586 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:31.104729 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:31.106660 (+ 1931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:31.106666 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:31.106694 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:31.107396 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.107411 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.110200 (+ 2789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.110207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.112973 (+ 2766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.112990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.115096 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.115102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.115294 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.115329 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 16:37:31.115422 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 16:37:31.115574 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28278,"mutex_wait_us":3,"thread_start_us":21367,"threads_started":1}
I20260928 16:37:31.116032 18725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.076449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57348 (local address 127.10.247.194:36775)
0928 16:37:31.105722 (+ 29273us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.105732 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.105763 (+ 31us) server_negotiation.cc:407] Connection header received
0928 16:37:31.105879 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.105886 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.106164 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.106401 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:31.107634 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.109984 (+ 2350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.113171 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.114560 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.114736 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.114903 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.115498 (+ 595us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.115717 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29078,"thread_start_us":29360,"threads_started":1}
I20260928 16:37:31.115579 18625 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae3cdc77c1842a2840e310574fddb0e (DEFAULT_TABLE table=test-table [id=9f28f3879e3143c2aa99fcfcd491f9fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:31.117659 18625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae3cdc77c1842a2840e310574fddb0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:31.118175 18334 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae3cdc77c1842a2840e310574fddb0e (DEFAULT_TABLE table=test-table [id=9f28f3879e3143c2aa99fcfcd491f9fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:31.119827 18334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae3cdc77c1842a2840e310574fddb0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:31.120748 18483 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae3cdc77c1842a2840e310574fddb0e (DEFAULT_TABLE table=test-table [id=9f28f3879e3143c2aa99fcfcd491f9fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:31.122277 18483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae3cdc77c1842a2840e310574fddb0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:31.133890 18727 tablet_bootstrap.cc:492] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: Bootstrap starting.
I20260928 16:37:31.137996 18727 tablet_bootstrap.cc:654] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:31.138295 18728 tablet_bootstrap.cc:492] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: Bootstrap starting.
I20260928 16:37:31.140381 18727 log.cc:824] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:31.141309 18729 tablet_bootstrap.cc:492] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: Bootstrap starting.
I20260928 16:37:31.143615 18728 tablet_bootstrap.cc:654] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:31.145366 18728 log.cc:824] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:31.145457 18729 tablet_bootstrap.cc:654] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:31.147001 18729 log.cc:824] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:31.149102 18727 tablet_bootstrap.cc:492] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: No bootstrap required, opened a new log
I20260928 16:37:31.149544 18727 ts_tablet_manager.cc:1397] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260928 16:37:31.150112 18728 tablet_bootstrap.cc:492] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: No bootstrap required, opened a new log
I20260928 16:37:31.150112 18729 tablet_bootstrap.cc:492] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: No bootstrap required, opened a new log
I20260928 16:37:31.150393 18729 ts_tablet_manager.cc:1397] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260928 16:37:31.150393 18728 ts_tablet_manager.cc:1397] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260928 16:37:31.163095 18727 raft_consensus.cc:348] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.163228 18729 raft_consensus.cc:348] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.163733 18727 raft_consensus.cc:374] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:31.163748 18729 raft_consensus.cc:374] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:31.163874 18727 raft_consensus.cc:729] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6b9625bcac486191f88591d1094bf6, State: Initialized, Role: FOLLOWER
I20260928 16:37:31.163923 18729 raft_consensus.cc:729] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76a12c7a96564b11a3daadc9a95fa1f6, State: Initialized, Role: FOLLOWER
I20260928 16:37:31.164695 18729 consensus_queue.cc:260] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.164717 18727 consensus_queue.cc:260] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.164966 18728 raft_consensus.cc:348] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.165541 18728 raft_consensus.cc:374] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:31.165666 18728 raft_consensus.cc:729] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e708b3522244403a966782fd941ddf23, State: Initialized, Role: FOLLOWER
I20260928 16:37:31.166440 18728 consensus_queue.cc:260] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.167781 18727 ts_tablet_manager.cc:1428] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: Time spent starting tablet: real 0.018s user 0.012s sys 0.004s
I20260928 16:37:31.168111 18691 heartbeater.cc:503] Master 127.10.247.254:44723 was elected leader, sending a full tablet report...
I20260928 16:37:31.168138 18549 heartbeater.cc:503] Master 127.10.247.254:44723 was elected leader, sending a full tablet report...
I20260928 16:37:31.173370 18729 ts_tablet_manager.cc:1428] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: Time spent starting tablet: real 0.023s user 0.010s sys 0.008s
I20260928 16:37:31.179864 18728 ts_tablet_manager.cc:1428] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: Time spent starting tablet: real 0.029s user 0.015s sys 0.012s
W20260928 16:37:31.206084 18693 tablet.cc:2369] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:37:31.257517 18551 tablet.cc:2369] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:37:31.274317 18402 tablet.cc:2369] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:37:31.400218 18734 raft_consensus.cc:482] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:31.400717 18734 raft_consensus.cc:504] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.403030 18734 leader_election.cc:288] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775), e708b3522244403a966782fd941ddf23 (127.10.247.193:42405)
I20260928 16:37:31.404249 18733 raft_consensus.cc:482] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:31.404763 18733 raft_consensus.cc:504] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.408397 18733 leader_election.cc:288] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135), e708b3522244403a966782fd941ddf23 (127.10.247.193:42405)
I20260928 16:37:31.411944 18726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.403946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:35799 (local address 127.10.247.193:42405)
0928 16:37:31.404304 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.404309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.404328 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:37:31.404411 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.404416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.404496 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.404587 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:31.405180 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.406434 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.409193 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.410654 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.410857 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.410955 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.411335 (+ 380us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.411505 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260928 16:37:31.412081 18697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.403327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:42405 (local address 127.10.247.195:35799)
0928 16:37:31.403818 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:31.403836 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:31.403951 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:31.404734 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:31.404740 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:31.404771 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:31.404991 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.405003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.406644 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.406650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.408932 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.408951 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.410870 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.410878 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.411121 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.411169 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 16:37:31.411300 (+ 131us) client_negotiation.cc:240] Negotiation successful
0928 16:37:31.411511 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":253}
I20260928 16:37:31.413633 18354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae3cdc77c1842a2840e310574fddb0e" candidate_uuid: "9c6b9625bcac486191f88591d1094bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e708b3522244403a966782fd941ddf23" is_pre_election: true
I20260928 16:37:31.414455 18354 raft_consensus.cc:2454] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c6b9625bcac486191f88591d1094bf6 in term 0.
I20260928 16:37:31.415589 18580 leader_election.cc:302] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c6b9625bcac486191f88591d1094bf6, e708b3522244403a966782fd941ddf23; no voters:
I20260928 16:37:31.415839 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.403285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:36775 (local address 127.10.247.195:43339)
0928 16:37:31.403492 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:31.403517 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:31.403687 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:31.404427 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:31.404432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:31.404460 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:31.404740 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.404754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.406662 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.406666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.409221 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.409235 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.414540 (+ 5305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.414549 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.415004 (+ 455us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.415056 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 16:37:31.415138 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 16:37:31.415326 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":520}
I20260928 16:37:31.416232 18734 raft_consensus.cc:2781] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:31.416385 18734 raft_consensus.cc:482] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:31.416481 18734 raft_consensus.cc:3037] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:31.417023 18725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.403653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:43339 (local address 127.10.247.194:36775)
0928 16:37:31.403892 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.403898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.403920 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:37:31.404011 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.404016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.404105 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.404199 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:31.404904 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.406500 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.409421 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.413853 (+ 4432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.416174 (+ 2321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.416269 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.416330 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.416508 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260928 16:37:31.419382 18503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae3cdc77c1842a2840e310574fddb0e" candidate_uuid: "9c6b9625bcac486191f88591d1094bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" is_pre_election: true
I20260928 16:37:31.420014 18503 raft_consensus.cc:2454] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c6b9625bcac486191f88591d1094bf6 in term 0.
I20260928 16:37:31.420218 18734 raft_consensus.cc:504] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.421084 18734 leader_election.cc:288] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [CANDIDATE]: Term 1 election: Requested vote from peers 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775), e708b3522244403a966782fd941ddf23 (127.10.247.193:42405)
I20260928 16:37:31.421702 18503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae3cdc77c1842a2840e310574fddb0e" candidate_uuid: "9c6b9625bcac486191f88591d1094bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76a12c7a96564b11a3daadc9a95fa1f6"
I20260928 16:37:31.421911 18503 raft_consensus.cc:3037] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:31.421862 18354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae3cdc77c1842a2840e310574fddb0e" candidate_uuid: "9c6b9625bcac486191f88591d1094bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e708b3522244403a966782fd941ddf23"
I20260928 16:37:31.422127 18354 raft_consensus.cc:3037] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:31.423302 18736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.408492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:60111 (local address 127.10.247.193:42405)
0928 16:37:31.410184 (+ 1692us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.410190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.413473 (+ 3283us) server_negotiation.cc:407] Connection header received
0928 16:37:31.413801 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.413812 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.413899 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.413987 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:31.415223 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.417239 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.420315 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.421722 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.421888 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.421978 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.422732 (+ 754us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.422908 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1508,"thread_start_us":1391,"threads_started":1}
I20260928 16:37:31.425182 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.408053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:42405 (local address 127.10.247.194:60111)
0928 16:37:31.413354 (+ 5301us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:31.413377 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:31.413619 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:31.414377 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:31.414382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:31.414419 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:31.415000 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.415015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.417578 (+ 2563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.417583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.420072 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.420093 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.422278 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.422285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.422486 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.422548 (+ 62us) client_negotiation.cc:769] Sending connection context
0928 16:37:31.423505 (+ 957us) client_negotiation.cc:240] Negotiation successful
0928 16:37:31.423684 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5115,"thread_start_us":9495,"threads_started":1}
I20260928 16:37:31.425352 18724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.408803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:44003 (local address 127.10.247.195:45135)
0928 16:37:31.409132 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.409139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.417978 (+ 8839us) server_negotiation.cc:407] Connection header received
0928 16:37:31.418261 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.418266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.418361 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.418437 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:31.419383 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.420398 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.422952 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.424186 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.424370 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.424449 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.424956 (+ 507us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.425122 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260928 16:37:31.426203 18737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.408142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:45135 (local address 127.10.247.194:44003)
0928 16:37:31.417876 (+ 9734us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:31.417895 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:31.418090 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:31.418686 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:31.418690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:31.418720 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:31.419197 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.419211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.420584 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.420589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.422755 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:31.422770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.424460 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.424466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:31.424741 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.424774 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:37:31.425554 (+ 780us) client_negotiation.cc:240] Negotiation successful
0928 16:37:31.425729 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9594,"thread_start_us":5828,"threads_started":1}
I20260928 16:37:31.426275 18353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae3cdc77c1842a2840e310574fddb0e" candidate_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e708b3522244403a966782fd941ddf23" is_pre_election: true
I20260928 16:37:31.426918 18503 raft_consensus.cc:2454] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c6b9625bcac486191f88591d1094bf6 in term 1.
I20260928 16:37:31.427459 18577 leader_election.cc:302] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76a12c7a96564b11a3daadc9a95fa1f6, 9c6b9625bcac486191f88591d1094bf6; no voters:
I20260928 16:37:31.427372 18645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae3cdc77c1842a2840e310574fddb0e" candidate_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6b9625bcac486191f88591d1094bf6" is_pre_election: true
I20260928 16:37:31.427994 18645 raft_consensus.cc:2379] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76a12c7a96564b11a3daadc9a95fa1f6 in current term 1: Already voted for candidate 9c6b9625bcac486191f88591d1094bf6 in this term.
I20260928 16:37:31.428231 18354 raft_consensus.cc:2454] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c6b9625bcac486191f88591d1094bf6 in term 1.
I20260928 16:37:31.428303 18734 raft_consensus.cc:2781] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:31.428987 18437 leader_election.cc:302] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76a12c7a96564b11a3daadc9a95fa1f6; no voters: 9c6b9625bcac486191f88591d1094bf6, e708b3522244403a966782fd941ddf23
I20260928 16:37:31.429244 18734 raft_consensus.cc:686] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 1 LEADER]: Becoming Leader. State: Replica: 9c6b9625bcac486191f88591d1094bf6, State: Running, Role: LEADER
I20260928 16:37:31.429426 18733 raft_consensus.cc:2726] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260928 16:37:31.430169 18734 consensus_queue.cc:237] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } }
I20260928 16:37:31.436424 17429 catalog_manager.cc:5495] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195). New cstate: current_term: 1 leader_uuid: "9c6b9625bcac486191f88591d1094bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } health_report { overall_health: UNKNOWN } } }
I20260928 16:37:31.467438 18724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.460067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.10.247.195:45135)
0928 16:37:31.460297 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.460304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.460388 (+ 84us) server_negotiation.cc:407] Connection header received
0928 16:37:31.460716 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.460723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.460824 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.460982 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:37:31.461419 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.462836 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.464360 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.465151 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.465283 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.465943 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:37:31.466692 (+ 749us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:37:31.466854 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.466956 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.467072 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 16:37:31.478559 18354 raft_consensus.cc:1261] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Refusing update from remote peer 9c6b9625bcac486191f88591d1094bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:31.478637 18503 raft_consensus.cc:1261] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 FOLLOWER]: Refusing update from remote peer 9c6b9625bcac486191f88591d1094bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:31.479750 18734 consensus_queue.cc:1035] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:31.480177 18739 consensus_queue.cc:1035] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:31.496162 18743 mvcc.cc:205] Tried to move back new op lower bound from 7334352697242877952 to 7334352697062363136. Current Snapshot: MvccSnapshot[applied={T|T < 7334352697242877952}]
I20260928 16:37:31.519251 18744 mvcc.cc:205] Tried to move back new op lower bound from 7334352697242877952 to 7334352697062363136. Current Snapshot: MvccSnapshot[applied={T|T < 7334352697242877952 or (T in {7334352697242877952})}]
I20260928 16:37:31.522406 17427 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:31.522909 17427 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:31.533802 17427 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 989b1693d84c4f1497a967c16aa39701: Sending DeleteTablet for 3 replicas of tablet 8ae3cdc77c1842a2840e310574fddb0e
I20260928 16:37:31.535620 18483 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae3cdc77c1842a2840e310574fddb0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:31 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:57348
I20260928 16:37:31.537613 18625 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae3cdc77c1842a2840e310574fddb0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:31 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:35602
I20260928 16:37:31.537925 18334 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae3cdc77c1842a2840e310574fddb0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:31 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:50358
I20260928 16:37:31.548080 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 17397
I20260928 16:37:31.549592 17459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:31.560578 18753 tablet_replica.cc:330] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: stopping tablet replica
I20260928 16:37:31.561913 18751 tablet_replica.cc:330] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: stopping tablet replica
I20260928 16:37:31.564322 18753 raft_consensus.cc:2229] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:31.563838 18751 raft_consensus.cc:2229] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:31.566201 18751 raft_consensus.cc:2258] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:31.567287 18753 raft_consensus.cc:2258] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:31.569422 18751 ts_tablet_manager.cc:1905] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:31.572638 18752 tablet_replica.cc:330] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: stopping tablet replica
I20260928 16:37:31.573665 18752 raft_consensus.cc:2229] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:37:31.574895 18753 ts_tablet_manager.cc:1905] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:31.576025 18752 raft_consensus.cc:2258] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:31.582109 18752 ts_tablet_manager.cc:1905] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:31.584514 18751 ts_tablet_manager.cc:1918] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:31.584837 18751 log.cc:1196] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/8ae3cdc77c1842a2840e310574fddb0e
I20260928 16:37:31.590782 18753 ts_tablet_manager.cc:1918] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:31.591004 18753 log.cc:1196] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/8ae3cdc77c1842a2840e310574fddb0e
I20260928 16:37:31.591712 18753 ts_tablet_manager.cc:1939] T 8ae3cdc77c1842a2840e310574fddb0e P e708b3522244403a966782fd941ddf23: Deleting consensus metadata
I20260928 16:37:31.599895 18752 ts_tablet_manager.cc:1918] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:31.600201 18752 log.cc:1196] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/8ae3cdc77c1842a2840e310574fddb0e
I20260928 16:37:31.601038 18752 ts_tablet_manager.cc:1939] T 8ae3cdc77c1842a2840e310574fddb0e P 9c6b9625bcac486191f88591d1094bf6: Deleting consensus metadata
I20260928 16:37:31.602046 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.541423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.10.247.253:42395)
0928 16:37:31.577150 (+ 35727us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.577157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.577187 (+ 30us) server_negotiation.cc:407] Connection header received
0928 16:37:31.577284 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.577370 (+ 86us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.577499 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.580938 (+ 3439us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:31.581155 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.583231 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.585631 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.586425 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.586573 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.591176 (+ 4603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:31.591186 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:31.591191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:31.591249 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:31.595265 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.596491 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.596497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.596501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.596733 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.597255 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.597261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.597264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.597707 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:31.597981 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.601325 (+ 3344us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.601628 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35576,"thread_start_us":36156,"threads_started":1}
I20260928 16:37:31.605074 18751 ts_tablet_manager.cc:1939] T 8ae3cdc77c1842a2840e310574fddb0e P 76a12c7a96564b11a3daadc9a95fa1f6: Deleting consensus metadata
I20260928 16:37:31.616884 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.547536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37748 (local address 127.10.247.252:38941)
0928 16:37:31.576423 (+ 28887us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.576671 (+ 248us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.599372 (+ 22701us) server_negotiation.cc:407] Connection header received
0928 16:37:31.599718 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.599726 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.599834 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.599935 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:31.603894 (+ 3959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.605056 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.606659 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.607153 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.607281 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.610250 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:31.610257 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:31.610261 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:31.610311 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:31.613717 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.614478 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.614483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.614486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.614731 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.615123 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.615127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.615130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.615547 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:31.615806 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.616168 (+ 362us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.616434 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28510,"thread_start_us":28666,"threads_started":1}
I20260928 16:37:31.789741 17416 catalog_manager.cc:4841] TS e708b3522244403a966782fd941ddf23 (127.10.247.193:42405): tablet 8ae3cdc77c1842a2840e310574fddb0e (table test-table [id=9f28f3879e3143c2aa99fcfcd491f9fb]) successfully deleted
I20260928 16:37:31.789403 17413 catalog_manager.cc:4841] TS 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775): tablet 8ae3cdc77c1842a2840e310574fddb0e (table test-table [id=9f28f3879e3143c2aa99fcfcd491f9fb]) successfully deleted
I20260928 16:37:31.790067 17414 catalog_manager.cc:4841] TS 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135): tablet 8ae3cdc77c1842a2840e310574fddb0e (table test-table [id=9f28f3879e3143c2aa99fcfcd491f9fb]) successfully deleted
I20260928 16:37:31.790753 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 17397
I20260928 16:37:31.844811 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.828599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.10.247.252:38941)
0928 16:37:31.828948 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.828954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.829979 (+ 1025us) server_negotiation.cc:407] Connection header received
0928 16:37:31.830148 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.830154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.830258 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.830376 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:31.832490 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.833477 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.835400 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.836092 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.836232 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.839347 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:31.839352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:31.839355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:31.839393 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:31.842105 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.842731 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.842737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.842741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.842870 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.843229 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.843234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.843236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.843569 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:31.843817 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.844196 (+ 379us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.844429 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260928 16:37:31.846323 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:31.827413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.10.247.253:42395)
0928 16:37:31.827708 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:31.827716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:31.828868 (+ 1152us) server_negotiation.cc:407] Connection header received
0928 16:37:31.829265 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:31.829272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:31.829395 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:31.829489 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:31.831953 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.833121 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.834685 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:31.835155 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:31.835271 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:31.838887 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:31.838895 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:31.838900 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:31.838950 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:31.842758 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.843630 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.843636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.843640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.843805 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:31.844274 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:31.844280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:31.844283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:31.844723 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:31.844958 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:31.845584 (+ 626us) server_negotiation.cc:299] Negotiation successful
0928 16:37:31.845825 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260928 16:37:32.504237 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:32.503287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.193:43933)
0928 16:37:32.503833 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:32.503981 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":290,"threads_started":1}
W20260928 16:37:32.504715 18400 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:44723 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
I20260928 16:37:32.514535 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:32.513714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.195:35393)
0928 16:37:32.514199 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:32.514340 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":247,"threads_started":1}
W20260928 16:37:32.515033 18691 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:44723 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
I20260928 16:37:32.525476 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:32.524558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.194:44297)
0928 16:37:32.525135 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:32.525280 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":264,"threads_started":1}
W20260928 16:37:32.525969 18549 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:44723 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
I20260928 16:37:33.033195 18767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 989b1693d84c4f1497a967c16aa39701)
I20260928 16:37:33.033491 18768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 989b1693d84c4f1497a967c16aa39701)
I20260928 16:37:33.033594 18767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } }
I20260928 16:37:33.033728 18768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } }
I20260928 16:37:33.034718 18768 leader_election.cc:288] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 989b1693d84c4f1497a967c16aa39701 (127.10.247.254:44723), 510d281a5b5740d2ab8ddceacbd631d0 (127.10.247.252:38941)
I20260928 16:37:33.035321 18767 leader_election.cc:288] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 989b1693d84c4f1497a967c16aa39701 (127.10.247.254:44723), 41a40f5a3199491db32d561d5d2caad9 (127.10.247.253:42395)
I20260928 16:37:33.036576 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.035452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34662)
0928 16:37:33.036032 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.036223 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":274,"threads_started":1}
W20260928 16:37:33.037086 18204 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
I20260928 16:37:33.046314 18771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.045311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34670)
0928 16:37:33.045888 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.046054 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446}
W20260928 16:37:33.046561 18204 leader_election.cc:334] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 989b1693d84c4f1497a967c16aa39701 (127.10.247.254:44723): Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
I20260928 16:37:33.053512 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.035639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:42395 (local address 127.0.0.1:59126)
0928 16:37:33.042388 (+ 6749us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.042422 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:33.042708 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:33.046604 (+ 3896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:33.046610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:33.046668 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:33.046927 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.046940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.048850 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.048857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.050969 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.050987 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.052550 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.052557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.052795 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.052835 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 16:37:33.052937 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 16:37:33.053127 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6582,"thread_start_us":9987,"threads_started":1}
I20260928 16:37:33.053736 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.034985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34654)
0928 16:37:33.053129 (+ 18144us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.053391 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18009,"thread_start_us":17844,"threads_started":1}
I20260928 16:37:33.054451 18772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.036092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.10.247.253:42395)
0928 16:37:33.045125 (+ 9033us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.045132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.045155 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:37:33.045249 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.045255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.045357 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.047104 (+ 1747us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:33.047443 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.048660 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.051203 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.052365 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.052544 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.052662 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.053968 (+ 1306us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.054138 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8860,"thread_start_us":9313,"threads_started":1}
I20260928 16:37:33.055269 18039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "510d281a5b5740d2ab8ddceacbd631d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "41a40f5a3199491db32d561d5d2caad9" is_pre_election: true
W20260928 16:37:33.055428 18017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
I20260928 16:37:33.055766 18039 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 510d281a5b5740d2ab8ddceacbd631d0 in term 1.
I20260928 16:37:33.056680 18201 leader_election.cc:302] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41a40f5a3199491db32d561d5d2caad9, 510d281a5b5740d2ab8ddceacbd631d0; no voters: 989b1693d84c4f1497a967c16aa39701
I20260928 16:37:33.057381 18767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 16:37:33.057580 18767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 989b1693d84c4f1497a967c16aa39701)
I20260928 16:37:33.057690 18767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 1 FOLLOWER]: Advancing to term 2
I20260928 16:37:33.061121 18776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.059855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34688)
0928 16:37:33.060584 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.060796 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":384,"threads_started":1}
W20260928 16:37:33.061393 18017 leader_election.cc:334] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 989b1693d84c4f1497a967c16aa39701 (127.10.247.254:44723): Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
I20260928 16:37:33.061595 18767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } }
I20260928 16:37:33.062402 18767 leader_election.cc:288] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [CANDIDATE]: Term 2 election: Requested vote from peers 989b1693d84c4f1497a967c16aa39701 (127.10.247.254:44723), 41a40f5a3199491db32d561d5d2caad9 (127.10.247.253:42395)
I20260928 16:37:33.062984 18039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "510d281a5b5740d2ab8ddceacbd631d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "41a40f5a3199491db32d561d5d2caad9"
I20260928 16:37:33.063313 18039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 1 FOLLOWER]: Advancing to term 2
I20260928 16:37:33.063813 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.062617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34692)
0928 16:37:33.062797 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.063590 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 16:37:33.064953 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.064497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34704)
0928 16:37:33.064697 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.064796 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260928 16:37:33.065194 18204 leader_election.cc:334] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 989b1693d84c4f1497a967c16aa39701 (127.10.247.254:44723): Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
I20260928 16:37:33.066073 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.053634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.10.247.252:38941)
0928 16:37:33.059020 (+ 5386us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.059026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.059057 (+ 31us) server_negotiation.cc:407] Connection header received
0928 16:37:33.059137 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.059142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.059237 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.060023 (+ 786us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:33.060340 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.061430 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.063705 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.065043 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.065234 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.065314 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.065677 (+ 363us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.065789 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5200,"thread_start_us":5013,"threads_started":1}
I20260928 16:37:33.066123 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.055077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:38941 (local address 127.0.0.1:57656)
0928 16:37:33.056029 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.056049 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:33.056185 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:33.059437 (+ 3252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:33.059444 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:33.059489 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:33.059844 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.059858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.061655 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.061662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.063541 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.063557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.065270 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.065277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.065493 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.065547 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 16:37:33.065653 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 16:37:33.065818 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839}
I20260928 16:37:33.066674 18039 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 510d281a5b5740d2ab8ddceacbd631d0 in term 2.
I20260928 16:37:33.066828 18228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "41a40f5a3199491db32d561d5d2caad9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "510d281a5b5740d2ab8ddceacbd631d0" is_pre_election: true
I20260928 16:37:33.067065 18201 leader_election.cc:302] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41a40f5a3199491db32d561d5d2caad9, 510d281a5b5740d2ab8ddceacbd631d0; no voters: 989b1693d84c4f1497a967c16aa39701
I20260928 16:37:33.067297 18228 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41a40f5a3199491db32d561d5d2caad9 in current term 2: Already voted for candidate 510d281a5b5740d2ab8ddceacbd631d0 in this term.
I20260928 16:37:33.067494 18767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 16:37:33.067915 18017 leader_election.cc:302] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41a40f5a3199491db32d561d5d2caad9; no voters: 510d281a5b5740d2ab8ddceacbd631d0, 989b1693d84c4f1497a967c16aa39701
I20260928 16:37:33.067986 18767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [term 2 LEADER]: Becoming Leader. State: Replica: 510d281a5b5740d2ab8ddceacbd631d0, State: Running, Role: LEADER
I20260928 16:37:33.068205 18768 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:37:33.068485 18767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } }
I20260928 16:37:33.070943 18777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 510d281a5b5740d2ab8ddceacbd631d0. Latest consensus state: current_term: 2 leader_uuid: "510d281a5b5740d2ab8ddceacbd631d0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:33.071410 18777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:33.072154 18779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:33.077035 18779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:33.078115 18779 catalog_manager.cc:1259] Loaded cluster ID: 0526e8f1799a42149a3bbf27fb8d4fc2
I20260928 16:37:33.078197 18779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:33.079551 18779 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:33.080489 18779 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Loaded TSK: 0
I20260928 16:37:33.081633 18779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:37:33.107968 18217 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:37:33.116362 18217 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:37:33.121773 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.121214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34730)
0928 16:37:33.121414 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.121549 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 16:37:33.121937 18039 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [term 2 FOLLOWER]: Refusing update from remote peer 510d281a5b5740d2ab8ddceacbd631d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260928 16:37:33.123059 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.122612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34742)
0928 16:37:33.122779 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.122875 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260928 16:37:33.122879 18777 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260928 16:37:33.123421 18204 consensus_peers.cc:487] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 -> Peer 989b1693d84c4f1497a967c16aa39701 (127.10.247.254:44723): Couldn't send request to peer 989b1693d84c4f1497a967c16aa39701. Status: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 16:37:33.126272 18768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 510d281a5b5740d2ab8ddceacbd631d0. Latest consensus state: current_term: 2 leader_uuid: "510d281a5b5740d2ab8ddceacbd631d0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:33.126570 18768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:37:33.128341 18767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "510d281a5b5740d2ab8ddceacbd631d0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:33.128782 18767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:33.128839 18781 mvcc.cc:205] Tried to move back new op lower bound from 7334352703978459136 to 7334352703773179904. Current Snapshot: MvccSnapshot[applied={T|T < 7334352703978459136}]
I20260928 16:37:33.129740 18768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "510d281a5b5740d2ab8ddceacbd631d0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "989b1693d84c4f1497a967c16aa39701" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44723 } } peers { permanent_uuid: "41a40f5a3199491db32d561d5d2caad9" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 42395 } attrs { promote: false } } peers { permanent_uuid: "510d281a5b5740d2ab8ddceacbd631d0" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 38941 } attrs { promote: false } } }
I20260928 16:37:33.130050 18768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a40f5a3199491db32d561d5d2caad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:37:33.148898 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.139021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:42405 (local address 127.0.0.1:49904)
0928 16:37:33.139506 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.139529 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:33.139654 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:33.140118 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:33.140123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:33.140150 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:33.140406 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.140418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.143220 (+ 2802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.143227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.145427 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.145448 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.147883 (+ 2435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.147890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.148151 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.148201 (+ 50us) client_negotiation.cc:769] Sending connection context
0928 16:37:33.148287 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 16:37:33.148486 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311}
I20260928 16:37:33.149971 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.139151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49904 (local address 127.10.247.193:42405)
0928 16:37:33.139717 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.139724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.139747 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:37:33.139839 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.139845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.139936 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.140565 (+ 629us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:33.140794 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.143016 (+ 2222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.145686 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.147661 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.149341 (+ 1680us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.149429 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.149489 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.149649 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":1396,"threads_started":1}
I20260928 16:37:33.150938 18334 tablet_service.cc:1467] Processing CreateTablet for tablet 24f105e6e5ae46569c4cb0bfba861834 (DEFAULT_TABLE table=test-table [id=1c198a53947946e987093ce66ae2a47d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:33.151503 18334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24f105e6e5ae46569c4cb0bfba861834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:33.158568 18790 tablet_bootstrap.cc:492] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: Bootstrap starting.
I20260928 16:37:33.159737 18771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.139396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:45135 (local address 127.0.0.1:53802)
0928 16:37:33.139987 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.140003 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:33.140113 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:33.152512 (+ 12399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:33.152520 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:33.152555 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:33.152959 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.152969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.154793 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.154799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.156936 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.156953 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.158906 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.158912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.159091 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.159132 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 16:37:33.159210 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 16:37:33.159365 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482}
I20260928 16:37:33.160744 18790 tablet_bootstrap.cc:654] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:33.160854 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.140366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.10.247.195:45135)
0928 16:37:33.151959 (+ 11593us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.151965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.151988 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:37:33.152081 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.152087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.152190 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.153125 (+ 935us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:33.153366 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.154614 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.157154 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.158502 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.160153 (+ 1651us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.160236 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.160296 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.160446 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11435,"thread_start_us":11920,"threads_started":1}
I20260928 16:37:33.161736 18625 tablet_service.cc:1467] Processing CreateTablet for tablet 24f105e6e5ae46569c4cb0bfba861834 (DEFAULT_TABLE table=test-table [id=1c198a53947946e987093ce66ae2a47d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:33.162254 18625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24f105e6e5ae46569c4cb0bfba861834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:33.162827 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.140068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36818 (local address 127.10.247.194:36775)
0928 16:37:33.155161 (+ 15093us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.155166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.155184 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:37:33.155253 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.155257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.155332 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.155428 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:33.156501 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.157616 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.160113 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.161373 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.161546 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.161626 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.162260 (+ 634us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.162479 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14963,"thread_start_us":14850,"threads_started":1}
I20260928 16:37:33.163682 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.139846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:36775 (local address 127.0.0.1:36818)
0928 16:37:33.148580 (+ 8734us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.148603 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:33.148769 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:33.155482 (+ 6713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:33.155488 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:33.155523 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:33.156260 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.156274 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.157762 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.157768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.159906 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.159923 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.161818 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.161824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.162026 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.162067 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 16:37:33.163040 (+ 973us) client_negotiation.cc:240] Negotiation successful
0928 16:37:33.163281 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8582,"thread_start_us":9022,"threads_started":1}
I20260928 16:37:33.164785 18483 tablet_service.cc:1467] Processing CreateTablet for tablet 24f105e6e5ae46569c4cb0bfba861834 (DEFAULT_TABLE table=test-table [id=1c198a53947946e987093ce66ae2a47d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:33.165304 18483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24f105e6e5ae46569c4cb0bfba861834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:33.169420 18792 tablet_bootstrap.cc:492] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: Bootstrap starting.
I20260928 16:37:33.169639 18790 tablet_bootstrap.cc:492] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: No bootstrap required, opened a new log
I20260928 16:37:33.169831 18790 ts_tablet_manager.cc:1397] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.010s
I20260928 16:37:33.170907 18790 raft_consensus.cc:348] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.171214 18790 raft_consensus.cc:374] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:33.171319 18790 raft_consensus.cc:729] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e708b3522244403a966782fd941ddf23, State: Initialized, Role: FOLLOWER
I20260928 16:37:33.171666 18792 tablet_bootstrap.cc:654] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:33.171646 18790 consensus_queue.cc:260] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.172137 18793 tablet_bootstrap.cc:492] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: Bootstrap starting.
I20260928 16:37:33.174207 18793 tablet_bootstrap.cc:654] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:33.177582 18790 ts_tablet_manager.cc:1428] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260928 16:37:33.179145 18793 tablet_bootstrap.cc:492] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: No bootstrap required, opened a new log
I20260928 16:37:33.179345 18793 ts_tablet_manager.cc:1397] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260928 16:37:33.179585 18792 tablet_bootstrap.cc:492] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: No bootstrap required, opened a new log
I20260928 16:37:33.180630 18793 raft_consensus.cc:348] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.180970 18793 raft_consensus.cc:374] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:33.181073 18793 raft_consensus.cc:729] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76a12c7a96564b11a3daadc9a95fa1f6, State: Initialized, Role: FOLLOWER
I20260928 16:37:33.181434 18793 consensus_queue.cc:260] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.182122 18792 ts_tablet_manager.cc:1397] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260928 16:37:33.183250 18792 raft_consensus.cc:348] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.183573 18792 raft_consensus.cc:374] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:33.183636 18793 ts_tablet_manager.cc:1428] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260928 16:37:33.183759 18792 raft_consensus.cc:729] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6b9625bcac486191f88591d1094bf6, State: Initialized, Role: FOLLOWER
I20260928 16:37:33.184074 18792 consensus_queue.cc:260] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.191730 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.183463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.194:49573)
0928 16:37:33.191126 (+ 7663us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.191336 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7503,"thread_start_us":7348,"threads_started":1}
I20260928 16:37:33.192598 18792 ts_tablet_manager.cc:1428] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: Time spent starting tablet: real 0.010s user 0.004s sys 0.004s
I20260928 16:37:33.194758 18548 heartbeater.cc:503] Master 127.10.247.252:38941 was elected leader, sending a full tablet report...
I20260928 16:37:33.196318 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.178410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.193:44735)
0928 16:37:33.195789 (+ 17379us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.195965 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17212,"thread_start_us":16487,"threads_started":1}
I20260928 16:37:33.197072 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.194137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.194:35637)
0928 16:37:33.196764 (+ 2627us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.196888 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2417}
I20260928 16:37:33.197926 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.197391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.193:35283)
0928 16:37:33.197666 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.197764 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260928 16:37:33.198113 18399 heartbeater.cc:503] Master 127.10.247.252:38941 was elected leader, sending a full tablet report...
I20260928 16:37:33.198397 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.192930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.195:47375)
0928 16:37:33.197836 (+ 4906us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.198068 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4775,"thread_start_us":4633,"threads_started":1}
I20260928 16:37:33.199810 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.198975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.195:34253)
0928 16:37:33.199557 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.199649 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":316,"threads_started":1}
I20260928 16:37:33.199915 18690 heartbeater.cc:503] Master 127.10.247.252:38941 was elected leader, sending a full tablet report...
I20260928 16:37:33.521656 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.521097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34756)
0928 16:37:33.521301 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.521429 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 16:37:33.522818 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.522402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34766)
0928 16:37:33.522547 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.522644 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 16:37:33.570919 18794 raft_consensus.cc:482] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:33.571237 18794 raft_consensus.cc:504] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.573484 18794 leader_election.cc:288] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135), 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775)
I20260928 16:37:33.580507 18798 raft_consensus.cc:482] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:33.580789 18798 raft_consensus.cc:504] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.581543 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.574160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:53031 (local address 127.10.247.194:36775)
0928 16:37:33.574428 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.574435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.574682 (+ 247us) server_negotiation.cc:407] Connection header received
0928 16:37:33.574850 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.574855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.574934 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.575025 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:33.575619 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.576685 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.579475 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.580350 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.580585 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.580697 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.581091 (+ 394us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.581244 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 16:37:33.581707 18798 leader_election.cc:288] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e708b3522244403a966782fd941ddf23 (127.10.247.193:42405), 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135)
I20260928 16:37:33.582310 18803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.573770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:36775 (local address 127.10.247.193:53031)
0928 16:37:33.574580 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.574596 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:33.574720 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:33.575090 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:33.575094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:33.575120 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:33.575465 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.575476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.576828 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.576833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.579312 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.579321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.580552 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.580557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.580836 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.580877 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 16:37:33.581706 (+ 829us) client_negotiation.cc:240] Negotiation successful
0928 16:37:33.581855 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":277,"thread_start_us":317,"threads_started":1}
I20260928 16:37:33.582636 18645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f105e6e5ae46569c4cb0bfba861834" candidate_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6b9625bcac486191f88591d1094bf6" is_pre_election: true
I20260928 16:37:33.582967 18645 raft_consensus.cc:2454] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76a12c7a96564b11a3daadc9a95fa1f6 in term 0.
I20260928 16:37:33.583128 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.573762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:45135 (local address 127.10.247.193:40017)
0928 16:37:33.573962 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.573982 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:33.574205 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:33.574843 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:33.574847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:33.574871 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:33.575173 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.575189 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.576728 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.576734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.579299 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:33.579317 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.582421 (+ 3104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.582428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:33.582612 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.582646 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 16:37:33.582740 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 16:37:33.582887 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":128}
I20260928 16:37:33.583254 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.574077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:40017 (local address 127.10.247.195:45135)
0928 16:37:33.574488 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.574494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.574516 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:37:33.574612 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.574616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.574703 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.574798 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:33.575369 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.576522 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.580802 (+ 4280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.582037 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.582183 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.582263 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.582789 (+ 526us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.582942 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260928 16:37:33.583674 18354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f105e6e5ae46569c4cb0bfba861834" candidate_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e708b3522244403a966782fd941ddf23" is_pre_election: true
I20260928 16:37:33.583913 18354 raft_consensus.cc:2454] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76a12c7a96564b11a3daadc9a95fa1f6 in term 0.
I20260928 16:37:33.583870 18503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f105e6e5ae46569c4cb0bfba861834" candidate_uuid: "e708b3522244403a966782fd941ddf23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" is_pre_election: true
I20260928 16:37:33.584122 18503 raft_consensus.cc:2454] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e708b3522244403a966782fd941ddf23 in term 0.
I20260928 16:37:33.584424 18439 leader_election.cc:302] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76a12c7a96564b11a3daadc9a95fa1f6, e708b3522244403a966782fd941ddf23; no voters:
I20260928 16:37:33.584779 18645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f105e6e5ae46569c4cb0bfba861834" candidate_uuid: "e708b3522244403a966782fd941ddf23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6b9625bcac486191f88591d1094bf6" is_pre_election: true
I20260928 16:37:33.584977 18798 raft_consensus.cc:2781] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:33.585021 18645 raft_consensus.cc:2454] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e708b3522244403a966782fd941ddf23 in term 0.
I20260928 16:37:33.584960 18287 leader_election.cc:302] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76a12c7a96564b11a3daadc9a95fa1f6, e708b3522244403a966782fd941ddf23; no voters:
I20260928 16:37:33.585101 18798 raft_consensus.cc:482] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:33.585191 18798 raft_consensus.cc:3037] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:33.585404 18794 raft_consensus.cc:2781] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:33.585534 18794 raft_consensus.cc:482] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:33.585621 18794 raft_consensus.cc:3037] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:33.587654 18798 raft_consensus.cc:504] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.587805 18794 raft_consensus.cc:504] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.588188 18798 leader_election.cc:288] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [CANDIDATE]: Term 1 election: Requested vote from peers e708b3522244403a966782fd941ddf23 (127.10.247.193:42405), 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135)
I20260928 16:37:33.588533 18794 leader_election.cc:288] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [CANDIDATE]: Term 1 election: Requested vote from peers 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135), 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775)
I20260928 16:37:33.588740 18645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f105e6e5ae46569c4cb0bfba861834" candidate_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6b9625bcac486191f88591d1094bf6"
I20260928 16:37:33.588955 18645 raft_consensus.cc:3037] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:33.588913 18644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f105e6e5ae46569c4cb0bfba861834" candidate_uuid: "e708b3522244403a966782fd941ddf23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6b9625bcac486191f88591d1094bf6"
I20260928 16:37:33.589584 18354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f105e6e5ae46569c4cb0bfba861834" candidate_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e708b3522244403a966782fd941ddf23"
I20260928 16:37:33.589756 18503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f105e6e5ae46569c4cb0bfba861834" candidate_uuid: "e708b3522244403a966782fd941ddf23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76a12c7a96564b11a3daadc9a95fa1f6"
I20260928 16:37:33.589886 18354 raft_consensus.cc:2379] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 76a12c7a96564b11a3daadc9a95fa1f6 in current term 1: Already voted for candidate e708b3522244403a966782fd941ddf23 in this term.
I20260928 16:37:33.590047 18503 raft_consensus.cc:2379] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e708b3522244403a966782fd941ddf23 in current term 1: Already voted for candidate 76a12c7a96564b11a3daadc9a95fa1f6 in this term.
I20260928 16:37:33.590585 18287 leader_election.cc:302] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e708b3522244403a966782fd941ddf23; no voters: 76a12c7a96564b11a3daadc9a95fa1f6, 9c6b9625bcac486191f88591d1094bf6
I20260928 16:37:33.590875 18794 raft_consensus.cc:2726] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260928 16:37:33.591713 18645 raft_consensus.cc:2454] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76a12c7a96564b11a3daadc9a95fa1f6 in term 1.
I20260928 16:37:33.592092 18437 leader_election.cc:302] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76a12c7a96564b11a3daadc9a95fa1f6, 9c6b9625bcac486191f88591d1094bf6; no voters: e708b3522244403a966782fd941ddf23
I20260928 16:37:33.592368 18798 raft_consensus.cc:2781] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:33.592876 18798 raft_consensus.cc:686] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 LEADER]: Becoming Leader. State: Replica: 76a12c7a96564b11a3daadc9a95fa1f6, State: Running, Role: LEADER
I20260928 16:37:33.593369 18798 consensus_queue.cc:237] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } }
I20260928 16:37:33.594029 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.593536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.194:54931)
0928 16:37:33.593727 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.593854 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 16:37:33.597275 18218 catalog_manager.cc:5495] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194). New cstate: current_term: 1 leader_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76a12c7a96564b11a3daadc9a95fa1f6" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 36775 } health_report { overall_health: HEALTHY } } }
I20260928 16:37:33.623652 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.617270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36826 (local address 127.10.247.194:36775)
0928 16:37:33.617469 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.617475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.617724 (+ 249us) server_negotiation.cc:407] Connection header received
0928 16:37:33.617999 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.618004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.618112 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.618262 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:37:33.618759 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.619641 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.621159 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.621640 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.621747 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.622188 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:37:33.622997 (+ 809us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:37:33.623148 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.623239 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.623330 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 16:37:33.628628 18354 raft_consensus.cc:1261] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Refusing update from remote peer 76a12c7a96564b11a3daadc9a95fa1f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:33.629015 18645 raft_consensus.cc:1261] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 1 FOLLOWER]: Refusing update from remote peer 76a12c7a96564b11a3daadc9a95fa1f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:33.629484 18804 consensus_queue.cc:1035] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e708b3522244403a966782fd941ddf23" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 42405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:33.629901 18798 consensus_queue.cc:1035] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c6b9625bcac486191f88591d1094bf6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:33.632925 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.632315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.193:43967)
0928 16:37:33.632495 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.632691 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260928 16:37:33.633108 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.632434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.195:40731)
0928 16:37:33.632751 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.632869 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260928 16:37:33.639798 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.639236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.195:34807)
0928 16:37:33.639485 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.639599 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260928 16:37:33.641324 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.640639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.194:54083)
0928 16:37:33.640996 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.641115 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226}
I20260928 16:37:33.644121 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.643606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.194:56891)
0928 16:37:33.643824 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.643915 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260928 16:37:33.646272 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.645673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.10.247.193:58351)
0928 16:37:33.645924 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:33.646042 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260928 16:37:33.657907 18218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:33.658250 18218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:33.665095 18218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 510d281a5b5740d2ab8ddceacbd631d0: Sending DeleteTablet for 3 replicas of tablet 24f105e6e5ae46569c4cb0bfba861834
I20260928 16:37:33.666523 18334 tablet_service.cc:1514] Processing DeleteTablet for tablet 24f105e6e5ae46569c4cb0bfba861834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:33 UTC) from {username='oryx', principal='oryx/127.10.247.252@KRBTEST.COM'} at 127.0.0.1:49904
I20260928 16:37:33.666702 18625 tablet_service.cc:1514] Processing DeleteTablet for tablet 24f105e6e5ae46569c4cb0bfba861834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:33 UTC) from {username='oryx', principal='oryx/127.10.247.252@KRBTEST.COM'} at 127.0.0.1:53802
I20260928 16:37:33.666776 18483 tablet_service.cc:1514] Processing DeleteTablet for tablet 24f105e6e5ae46569c4cb0bfba861834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:33 UTC) from {username='oryx', principal='oryx/127.10.247.252@KRBTEST.COM'} at 127.0.0.1:36818
I20260928 16:37:33.669219 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 18212
I20260928 16:37:33.683444 18819 tablet_replica.cc:330] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: stopping tablet replica
I20260928 16:37:33.683773 18819 raft_consensus.cc:2229] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:33.684195 18819 raft_consensus.cc:2258] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:33.686906 18818 tablet_replica.cc:330] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: stopping tablet replica
I20260928 16:37:33.687300 18818 raft_consensus.cc:2229] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:33.687675 18818 raft_consensus.cc:2258] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:33.687638 18819 ts_tablet_manager.cc:1905] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:33.688836 18818 ts_tablet_manager.cc:1905] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:33.693325 18820 tablet_replica.cc:330] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: stopping tablet replica
I20260928 16:37:33.693594 18820 raft_consensus.cc:2229] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:37:33.695020 18820 raft_consensus.cc:2258] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:33.695968 18818 ts_tablet_manager.cc:1918] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:33.695986 18819 ts_tablet_manager.cc:1918] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:33.696139 18819 log.cc:1196] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/24f105e6e5ae46569c4cb0bfba861834
I20260928 16:37:33.696151 18820 ts_tablet_manager.cc:1905] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:33.696216 18818 log.cc:1196] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/24f105e6e5ae46569c4cb0bfba861834
I20260928 16:37:33.696803 18818 ts_tablet_manager.cc:1939] T 24f105e6e5ae46569c4cb0bfba861834 P e708b3522244403a966782fd941ddf23: Deleting consensus metadata
I20260928 16:37:33.696887 18819 ts_tablet_manager.cc:1939] T 24f105e6e5ae46569c4cb0bfba861834 P 9c6b9625bcac486191f88591d1094bf6: Deleting consensus metadata
I20260928 16:37:33.698340 18202 catalog_manager.cc:4841] TS 9c6b9625bcac486191f88591d1094bf6 (127.10.247.195:45135): tablet 24f105e6e5ae46569c4cb0bfba861834 (table test-table [id=1c198a53947946e987093ce66ae2a47d]) successfully deleted
I20260928 16:37:33.698319 18204 catalog_manager.cc:4841] TS e708b3522244403a966782fd941ddf23 (127.10.247.193:42405): tablet 24f105e6e5ae46569c4cb0bfba861834 (table test-table [id=1c198a53947946e987093ce66ae2a47d]) successfully deleted
I20260928 16:37:33.703310 18820 ts_tablet_manager.cc:1918] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:33.703455 18820 log.cc:1196] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/24f105e6e5ae46569c4cb0bfba861834
I20260928 16:37:33.704018 18820 ts_tablet_manager.cc:1939] T 24f105e6e5ae46569c4cb0bfba861834 P 76a12c7a96564b11a3daadc9a95fa1f6: Deleting consensus metadata
I20260928 16:37:33.705487 18201 catalog_manager.cc:4841] TS 76a12c7a96564b11a3daadc9a95fa1f6 (127.10.247.194:36775): tablet 24f105e6e5ae46569c4cb0bfba861834 (table test-table [id=1c198a53947946e987093ce66ae2a47d]) successfully deleted
I20260928 16:37:33.706247 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.670512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49914 (local address 127.10.247.193:42405)
0928 16:37:33.677703 (+ 7191us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.677711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.678333 (+ 622us) server_negotiation.cc:407] Connection header received
0928 16:37:33.678475 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.678481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.678663 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.678803 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:33.682743 (+ 3940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.684342 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.692861 (+ 8519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.693595 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.693742 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.698274 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:33.698302 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:33.698328 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:33.698378 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:33.702542 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:33.703626 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:33.703638 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:33.703641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:33.703834 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:33.704368 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:33.704373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:33.704376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:33.704954 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:33.705315 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.705669 (+ 354us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.705908 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7026,"thread_start_us":6858,"threads_started":1}
I20260928 16:37:33.707000 18394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:33.877434 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 18212
I20260928 16:37:33.896291 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 18411
I20260928 16:37:33.914835 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:33.897416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.10.247.194:36775)
0928 16:37:33.897615 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:33.897620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:33.897640 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:37:33.897835 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:33.897840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:33.897919 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:33.898067 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:33.900104 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.901107 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.903075 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:33.903572 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:33.903683 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:33.907435 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:33.907458 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:33.907480 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:33.907559 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:33.911311 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:33.912385 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:33.912395 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:33.912397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:33.912648 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:33.913127 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:33.913132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:33.913137 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:33.913649 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:33.913938 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:33.914302 (+ 364us) server_negotiation.cc:299] Negotiation successful
0928 16:37:33.914513 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 16:37:33.915784 18543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:34.070094 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 18411
I20260928 16:37:34.087255 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:34.086399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34772)
0928 16:37:34.086914 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:34.087029 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":254,"threads_started":1}
I20260928 16:37:34.088785 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:34.088350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44723 (local address 127.0.0.1:34784)
0928 16:37:34.088524 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:34.088636 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 16:37:34.090258 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 18561
I20260928 16:37:34.106158 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:34.091134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.10.247.195:45135)
0928 16:37:34.091687 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:34.091693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:34.091711 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:37:34.091779 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:34.091783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:34.091852 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:34.091994 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:34.093704 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:34.094556 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:34.095872 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:34.096349 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:34.096454 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:34.099445 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:34.099469 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:34.099491 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:34.099543 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:34.102791 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:34.103893 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:34.103904 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:34.103908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:34.104133 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:34.104503 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:34.104506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:34.104508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:34.104968 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:34.105269 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:34.105510 (+ 241us) server_negotiation.cc:299] Negotiation successful
0928 16:37:34.105732 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":273,"threads_started":1}
I20260928 16:37:34.106905 18685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:34.242894 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 18561
I20260928 16:37:34.264820 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 17893
I20260928 16:37:34.266219 18060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:34.400102 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 17893
I20260928 16:37:34.421530 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 17994
I20260928 16:37:34.422871 18249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:34.561848 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 17994
2026-09-28T16:37:34Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11142 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:37:34 dist-test-slave-9fn6 krb5kdc[18833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:37:34 dist-test-slave-9fn6 krb5kdc[18833](info): set up 2 sockets
Sep 28 16:37:34 dist-test-slave-9fn6 krb5kdc[18833](info): commencing operation
krb5kdc: starting...
W20260928 16:37:35.597905 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:37:35 dist-test-slave-9fn6 krb5kdc[18833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:37:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.254.keytab.
Entry for principal oryx/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.254.keytab.
I20260928 16:37:35.873402 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42873
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.254.keytab
--principal=oryx/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:42873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:36.281481 18846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:36.281886 18846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:36.282050 18846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:36.290087 18846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:36.290264 18846 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 16:37:36.290341 18846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:36.290433 18846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:36.290504 18846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:36.290598 18846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:36.302207 18846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:42873
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.254.keytab
--principal=oryx/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:42873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:36.304059 18846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:36.305943 18846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:36.320081 18851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:36.320012 18852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:36.320001 18854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:36.321106 18846 server_base.cc:1034] running on GCE node
I20260928 16:37:36.322152 18846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:36.324018 18846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:36.325270 18846 hybrid_clock.cc:648] HybridClock initialized: now 1790613456325191 us; error 43 us; skew 500 ppm
I20260928 16:37:36.325687 18846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:36 dist-test-slave-9fn6 krb5kdc[18833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613456, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:36.337157 18846 init.cc:377] Logged in from keytab as oryx/127.10.247.254@KRBTEST.COM (short username oryx)
I20260928 16:37:36.339174 18846 webserver.cc:460] Webserver started at http://127.10.247.254:46785/ using document root <none> and password file <none>
I20260928 16:37:36.340147 18846 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:36.340255 18846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:36.340685 18846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:36.344501 18846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "859e198a0a9c4586b149752c11f70033"
format_stamp: "Formatted at 2026-09-28 16:37:36 on dist-test-slave-9fn6"
server_key: "8202d37ce4a92360e378929c7d849c1c"
server_key_iv: "9311be9fa098850f429a739613eebaa4"
server_key_version: "encryptionkey@0"
I20260928 16:37:36.345511 18846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "859e198a0a9c4586b149752c11f70033"
format_stamp: "Formatted at 2026-09-28 16:37:36 on dist-test-slave-9fn6"
server_key: "8202d37ce4a92360e378929c7d849c1c"
server_key_iv: "9311be9fa098850f429a739613eebaa4"
server_key_version: "encryptionkey@0"
I20260928 16:37:36.351799 18846 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:37:36.356109 18861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:36.357785 18846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260928 16:37:36.357993 18846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "859e198a0a9c4586b149752c11f70033"
format_stamp: "Formatted at 2026-09-28 16:37:36 on dist-test-slave-9fn6"
server_key: "8202d37ce4a92360e378929c7d849c1c"
server_key_iv: "9311be9fa098850f429a739613eebaa4"
server_key_version: "encryptionkey@0"
I20260928 16:37:36.358242 18846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:36.390618 18846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:36.397136 18846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:36.397459 18846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:36.420946 18846 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:42873
I20260928 16:37:36.420959 18923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:42873 every 8 connection(s)
I20260928 16:37:36.422854 18846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:36.424731 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 18846
I20260928 16:37:36.424980 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:36.425899 11231 external_mini_cluster.cc:1428] Setting key a828f956ce83094ac952b8b657aeb636
I20260928 16:37:36.429217 18924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 16:37:36 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:36.445302 18924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033: Bootstrap starting.
I20260928 16:37:36.450421 18924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:36.452039 18924 log.cc:824] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:36.455116 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:36.428647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.10.247.254:42873)
0928 16:37:36.429615 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:36.429630 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:36.429682 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:37:36.431622 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:36.431653 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:36.432158 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:36.432649 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:36.434882 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:36.436538 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:36.438549 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:36.439385 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:36.439572 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:36.445196 (+ 5624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:36.445211 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:36.445239 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:36.445311 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:36.449605 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:36.450555 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:36.450566 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:36.450577 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:36.450824 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:36.451274 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:36.451278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:36.451281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:36.452146 (+ 865us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:36.452482 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:36.453022 (+ 540us) server_negotiation.cc:299] Negotiation successful
0928 16:37:36.453507 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":430,"threads_started":1}
I20260928 16:37:36.456436 18924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033: No bootstrap required, opened a new log
I20260928 16:37:36.464950 18924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859e198a0a9c4586b149752c11f70033" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42873 } }
I20260928 16:37:36.465430 18924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:36.465540 18924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 859e198a0a9c4586b149752c11f70033, State: Initialized, Role: FOLLOWER
I20260928 16:37:36.466212 18924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859e198a0a9c4586b149752c11f70033" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42873 } }
I20260928 16:37:36.466463 18924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:36.466611 18924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:36.466797 18924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:36.469341 18924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859e198a0a9c4586b149752c11f70033" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42873 } }
I20260928 16:37:36.469924 18924 leader_election.cc:302] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 859e198a0a9c4586b149752c11f70033; no voters:
I20260928 16:37:36.470547 18924 leader_election.cc:288] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:36.470696 18929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:36.471419 18929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [term 1 LEADER]: Becoming Leader. State: Replica: 859e198a0a9c4586b149752c11f70033, State: Running, Role: LEADER
I20260928 16:37:36.471985 18929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859e198a0a9c4586b149752c11f70033" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42873 } }
I20260928 16:37:36.472860 18924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:36.475140 18931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 859e198a0a9c4586b149752c11f70033. Latest consensus state: current_term: 1 leader_uuid: "859e198a0a9c4586b149752c11f70033" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859e198a0a9c4586b149752c11f70033" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42873 } } }
I20260928 16:37:36.475113 18930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "859e198a0a9c4586b149752c11f70033" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859e198a0a9c4586b149752c11f70033" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 42873 } } }
I20260928 16:37:36.475625 18931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:36.475626 18930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033 [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:36.482915 18935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:36.488227 18935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:36.498524 18935 catalog_manager.cc:1347] Generated new cluster ID: 5319f0f512f841cf9b03f1ca4386f68f
I20260928 16:37:36.498677 18935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:36.508010 18935 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:36.510313 18935 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:36.523989 18935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033: Generated new TSK 0
I20260928 16:37:36.525624 18935 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab.
Entry for principal oryx/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab.
I20260928 16:37:36.608819 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42873
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab
--principal=oryx/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:37.004537 18950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:37.005016 18950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:37.005187 18950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:37.013656 18950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:37.013849 18950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:37.013993 18950 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 16:37:37.014127 18950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:37:37.026083 18950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab
--principal=oryx/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.247.254:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:37.027920 18950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:37.029711 18950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:37.046489 18956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:37.046607 18955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:37.047014 18958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:37.047429 18950 server_base.cc:1034] running on GCE node
I20260928 16:37:37.048205 18950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:37.049445 18950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:37.050700 18950 hybrid_clock.cc:648] HybridClock initialized: now 1790613457050614 us; error 53 us; skew 500 ppm
I20260928 16:37:37.051100 18950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:37 dist-test-slave-9fn6 krb5kdc[18833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613457, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:37.054906 18950 init.cc:377] Logged in from keytab as oryx/127.10.247.193@KRBTEST.COM (short username oryx)
I20260928 16:37:37.056900 18950 webserver.cc:460] Webserver started at http://127.10.247.193:34845/ using document root <none> and password file <none>
I20260928 16:37:37.057806 18950 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:37.057927 18950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:37.058326 18950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:37.062157 18950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "28b94d34437b43869003e8f777246eb7"
format_stamp: "Formatted at 2026-09-28 16:37:37 on dist-test-slave-9fn6"
server_key: "fc714ccd22f2e4d7f2eda972dbf4e832"
server_key_iv: "b0aa97d584cbd0ca4c80f164fb72e1d2"
server_key_version: "encryptionkey@0"
I20260928 16:37:37.063189 18950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "28b94d34437b43869003e8f777246eb7"
format_stamp: "Formatted at 2026-09-28 16:37:37 on dist-test-slave-9fn6"
server_key: "fc714ccd22f2e4d7f2eda972dbf4e832"
server_key_iv: "b0aa97d584cbd0ca4c80f164fb72e1d2"
server_key_version: "encryptionkey@0"
I20260928 16:37:37.069667 18950 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260928 16:37:37.074178 18965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:37.075940 18950 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:37:37.076143 18950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "28b94d34437b43869003e8f777246eb7"
format_stamp: "Formatted at 2026-09-28 16:37:37 on dist-test-slave-9fn6"
server_key: "fc714ccd22f2e4d7f2eda972dbf4e832"
server_key_iv: "b0aa97d584cbd0ca4c80f164fb72e1d2"
server_key_version: "encryptionkey@0"
I20260928 16:37:37.076419 18950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:37.090785 18950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:37.095551 18950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:37.095865 18950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:37.099270 18950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:37.099390 18950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:37.099530 18950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:37.099611 18950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:37:37 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613457, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:37.125641 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:37.099874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:46791 (local address 127.10.247.254:42873)
0928 16:37:37.100754 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:37.100759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:37.102109 (+ 1350us) server_negotiation.cc:407] Connection header received
0928 16:37:37.104393 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:37.104399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:37.104503 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:37.104650 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:37.107333 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.108451 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.111161 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.111735 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.111893 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:37.117342 (+ 5449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:37.117351 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:37.117356 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:37.117423 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:37.121466 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:37.122477 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:37.122484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:37.122488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:37.122694 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:37.123210 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:37.123215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:37.123218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:37.123678 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:37.123855 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:37.124902 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0928 16:37:37.125217 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":593,"threads_started":1}
I20260928 16:37:37.127591 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:37.100798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42873 (local address 127.10.247.193:46791)
0928 16:37:37.101830 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:37.101886 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:37.104036 (+ 2150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:37.104994 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:37.105009 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:37.105966 (+ 957us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:37.107092 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:37.107132 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.108669 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.108673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:37.110966 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:37.110992 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.112060 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.112068 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:37.112322 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:37.113178 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:37.113227 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:37.116899 (+ 3672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:37.121688 (+ 4789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:37.121698 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:37.121718 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:37.122316 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:37.122824 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:37.122828 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:37.122830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:37.123100 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:37.123807 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:37.123816 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:37.124395 (+ 579us) client_negotiation.cc:769] Sending connection context
0928 16:37:37.124746 (+ 351us) client_negotiation.cc:240] Negotiation successful
0928 16:37:37.125485 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":423,"threads_started":1}
I20260928 16:37:37.149685 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:37.135411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:33849 (local address 127.10.247.254:42873)
0928 16:37:37.135610 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:37.135615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:37.136210 (+ 595us) server_negotiation.cc:407] Connection header received
0928 16:37:37.136409 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:37.136415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:37.136527 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:37.136656 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:37.137789 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.138573 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.139942 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.140426 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.140578 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:37.143479 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:37.143486 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:37.143489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:37.143536 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:37.146720 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:37.147459 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:37.147464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:37.147468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:37.147624 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:37.147984 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:37.147989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:37.147992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:37.148317 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:37.148480 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:37.148983 (+ 503us) server_negotiation.cc:299] Negotiation successful
0928 16:37:37.149248 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:37:37.150213 19049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:37.135427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42873 (local address 127.10.247.193:33849)
0928 16:37:37.136083 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:37.136103 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:37.136273 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:37.136712 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:37.136717 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:37.137279 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:37.137628 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:37.137639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.138714 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.138719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:37.139791 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:37.139805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.140770 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.140780 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:37.141013 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:37.141914 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:37.141943 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:37.143251 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:37.146862 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:37.146871 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:37.146876 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:37.147335 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:37.147736 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:37.147741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:37.147745 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:37.147881 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:37.148436 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:37.148442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:37.148702 (+ 260us) client_negotiation.cc:769] Sending connection context
0928 16:37:37.148853 (+ 151us) client_negotiation.cc:240] Negotiation successful
0928 16:37:37.149324 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":354,"threads_started":1}
I20260928 16:37:37.157078 18950 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:35365
I20260928 16:37:37.157127 19085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:35365 every 8 connection(s)
I20260928 16:37:37.158550 18950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:37.161132 19086 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42873
I20260928 16:37:37.161388 19086 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:37.162225 19086 heartbeater.cc:511] Master 127.10.247.254:42873 requested a full tablet report, sending...
I20260928 16:37:37.163388 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 18950
I20260928 16:37:37.163630 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:37:37.164515 11231 external_mini_cluster.cc:1428] Setting key d65b66e708d8cefdd8c78358f1dec218
I20260928 16:37:37.165210 18878 ts_manager.cc:194] Registered new tserver with Master: 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365)
I20260928 16:37:37.167389 18878 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:46791
WARNING: no policy specified for oryx/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab.
Entry for principal oryx/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab.
I20260928 16:37:37.224505 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42873
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab
--principal=oryx/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:37.609076 19090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:37.609462 19090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:37.609575 19090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:37.617718 19090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:37.617880 19090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:37.617985 19090 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 16:37:37.618074 19090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:37.629951 19090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab
--principal=oryx/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.247.254:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:37.632009 19090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:37.633859 19090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:37.649442 19095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:37.649415 19096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:37.649644 19098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:37.651087 19090 server_base.cc:1034] running on GCE node
I20260928 16:37:37.651947 19090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:37.653322 19090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:37.654579 19090 hybrid_clock.cc:648] HybridClock initialized: now 1790613457654496 us; error 59 us; skew 500 ppm
I20260928 16:37:37.654939 19090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:37 dist-test-slave-9fn6 krb5kdc[18833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613457, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:37.658927 19090 init.cc:377] Logged in from keytab as oryx/127.10.247.194@KRBTEST.COM (short username oryx)
I20260928 16:37:37.660943 19090 webserver.cc:460] Webserver started at http://127.10.247.194:42745/ using document root <none> and password file <none>
I20260928 16:37:37.661878 19090 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:37.662019 19090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:37.662359 19090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:37.665997 19090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "e328164a9271443eb58fc5eba50f268a"
format_stamp: "Formatted at 2026-09-28 16:37:37 on dist-test-slave-9fn6"
server_key: "bdb0b9c4d32911438ff1c558c117ac92"
server_key_iv: "c60663235490ad1410f00f3fdedc94c1"
server_key_version: "encryptionkey@0"
I20260928 16:37:37.666853 19090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "e328164a9271443eb58fc5eba50f268a"
format_stamp: "Formatted at 2026-09-28 16:37:37 on dist-test-slave-9fn6"
server_key: "bdb0b9c4d32911438ff1c558c117ac92"
server_key_iv: "c60663235490ad1410f00f3fdedc94c1"
server_key_version: "encryptionkey@0"
I20260928 16:37:37.672996 19090 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260928 16:37:37.677286 19105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:37.679087 19090 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260928 16:37:37.679251 19090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "e328164a9271443eb58fc5eba50f268a"
format_stamp: "Formatted at 2026-09-28 16:37:37 on dist-test-slave-9fn6"
server_key: "bdb0b9c4d32911438ff1c558c117ac92"
server_key_iv: "c60663235490ad1410f00f3fdedc94c1"
server_key_version: "encryptionkey@0"
I20260928 16:37:37.679482 19090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:37.711153 19090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:37.716832 19090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:37.717130 19090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:37.720482 19090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:37.720590 19090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:37.720777 19090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:37.720846 19090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:37:37 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613457, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:37.749818 19115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:37.722164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42873 (local address 127.10.247.194:45715)
0928 16:37:37.723186 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:37.723255 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:37.726270 (+ 3015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:37.727406 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:37.727422 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:37.728436 (+ 1014us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:37.729763 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:37.729811 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.731454 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.731460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:37.733704 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:37.733730 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.734669 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.734677 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:37.734937 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:37.735723 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:37.735750 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:37.739171 (+ 3421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:37.743708 (+ 4537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:37.743718 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:37.743738 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:37.744291 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:37.744812 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:37.744821 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:37.744825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:37.745174 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:37.745827 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:37.745840 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:37.746383 (+ 543us) client_negotiation.cc:769] Sending connection context
0928 16:37:37.746818 (+ 435us) client_negotiation.cc:240] Negotiation successful
0928 16:37:37.747556 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":362,"threads_started":1}
I20260928 16:37:37.750692 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:37.721364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:45715 (local address 127.10.247.254:42873)
0928 16:37:37.721898 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:37.721907 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:37.723414 (+ 1507us) server_negotiation.cc:407] Connection header received
0928 16:37:37.726755 (+ 3341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:37.726763 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:37.726879 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:37.726991 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:37.730139 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.731237 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.733896 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.734383 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.734515 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:37.739534 (+ 5019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:37.739541 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:37.739544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:37.739590 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:37.743515 (+ 3925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:37.744432 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:37.744439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:37.744444 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:37.744670 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:37.745321 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:37.745326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:37.745328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:37.745678 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:37.745852 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:37.750021 (+ 4169us) server_negotiation.cc:299] Negotiation successful
0928 16:37:37.750369 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":254,"threads_started":1}
I20260928 16:37:37.774206 19090 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:35667
I20260928 16:37:37.774263 19225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:35667 every 8 connection(s)
I20260928 16:37:37.774603 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:37.757425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:52847 (local address 127.10.247.254:42873)
0928 16:37:37.757677 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:37.757684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:37.758290 (+ 606us) server_negotiation.cc:407] Connection header received
0928 16:37:37.758505 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:37.758512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:37.758602 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:37.758720 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:37.760505 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.761622 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.763962 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.764541 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.764691 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:37.767461 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:37.767469 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:37.767475 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:37.767541 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:37.771042 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:37.772002 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:37.772007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:37.772011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:37.772148 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:37.772600 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:37.772627 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:37.772630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:37.773061 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:37.773232 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:37.773915 (+ 683us) server_negotiation.cc:299] Negotiation successful
0928 16:37:37.774234 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260928 16:37:37.774881 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:37.757479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42873 (local address 127.10.247.194:52847)
0928 16:37:37.758150 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:37.758174 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:37.758359 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:37.758786 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:37.758791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:37.759685 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:37.760287 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:37.760312 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.761822 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.761826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:37.763624 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:37.763649 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:37.764771 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:37.764782 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:37.764960 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:37.765790 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:37.765810 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:37.767133 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:37.771284 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:37.771293 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:37.771297 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:37.771834 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:37.772285 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:37.772291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:37.772293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:37.772446 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:37.773200 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:37.773205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:37.773488 (+ 283us) client_negotiation.cc:769] Sending connection context
0928 16:37:37.773763 (+ 275us) client_negotiation.cc:240] Negotiation successful
0928 16:37:37.774396 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":368,"threads_started":1}
I20260928 16:37:37.776269 19090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:37.778968 19226 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42873
I20260928 16:37:37.779237 19226 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:37.780054 19226 heartbeater.cc:511] Master 127.10.247.254:42873 requested a full tablet report, sending...
I20260928 16:37:37.781903 18878 ts_manager.cc:194] Registered new tserver with Master: e328164a9271443eb58fc5eba50f268a (127.10.247.194:35667)
I20260928 16:37:37.782928 18878 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:45715
I20260928 16:37:37.784761 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 19090
I20260928 16:37:37.784965 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:37:37.785753 11231 external_mini_cluster.cc:1428] Setting key 979a93eef9033b69a5dbef72eb3d86b8
WARNING: no policy specified for oryx/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab.
Entry for principal oryx/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab.
I20260928 16:37:37.844331 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:42873
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab
--principal=oryx/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:37:38.171164 19086 heartbeater.cc:503] Master 127.10.247.254:42873 was elected leader, sending a full tablet report...
W20260928 16:37:38.242408 19230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:38.242839 19230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:38.242955 19230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:38.250950 19230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:38.251114 19230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:38.251231 19230 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 16:37:38.251351 19230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:38.262888 19230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab
--principal=oryx/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.247.254:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:38.264703 19230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:38.266396 19230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:38.281853 19235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:38.281801 19236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:38.281771 19238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:38.282785 19230 server_base.cc:1034] running on GCE node
I20260928 16:37:38.283569 19230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:38.284722 19230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:38.285940 19230 hybrid_clock.cc:648] HybridClock initialized: now 1790613458285867 us; error 41 us; skew 500 ppm
I20260928 16:37:38.286294 19230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:38 dist-test-slave-9fn6 krb5kdc[18833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613458, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:38.289762 19230 init.cc:377] Logged in from keytab as oryx/127.10.247.195@KRBTEST.COM (short username oryx)
I20260928 16:37:38.291416 19230 webserver.cc:460] Webserver started at http://127.10.247.195:37381/ using document root <none> and password file <none>
I20260928 16:37:38.292263 19230 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:38.292373 19230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:38.292775 19230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:38.296123 19230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4"
format_stamp: "Formatted at 2026-09-28 16:37:38 on dist-test-slave-9fn6"
server_key: "043536d32f486350796d5ffda98a85e6"
server_key_iv: "edff20d8030ebe48e58511f46dfad9e6"
server_key_version: "encryptionkey@0"
I20260928 16:37:38.297083 19230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4"
format_stamp: "Formatted at 2026-09-28 16:37:38 on dist-test-slave-9fn6"
server_key: "043536d32f486350796d5ffda98a85e6"
server_key_iv: "edff20d8030ebe48e58511f46dfad9e6"
server_key_version: "encryptionkey@0"
I20260928 16:37:38.302856 19230 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260928 16:37:38.306725 19245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:38.308207 19230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:37:38.308372 19230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4"
format_stamp: "Formatted at 2026-09-28 16:37:38 on dist-test-slave-9fn6"
server_key: "043536d32f486350796d5ffda98a85e6"
server_key_iv: "edff20d8030ebe48e58511f46dfad9e6"
server_key_version: "encryptionkey@0"
I20260928 16:37:38.308650 19230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:38.337186 19230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:38.342388 19230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:38.342715 19230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:38.346127 19230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:38.346248 19230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:38.346393 19230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:38.346457 19230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:37:38 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613458, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:38.374706 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.347347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:38497 (local address 127.10.247.254:42873)
0928 16:37:38.347927 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.347935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.349121 (+ 1186us) server_negotiation.cc:407] Connection header received
0928 16:37:38.351796 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.351804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.351929 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.352030 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:38.355373 (+ 3343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.356776 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.359571 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.360189 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.360351 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.366107 (+ 5756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:38.366117 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:38.366122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:38.366176 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:38.369804 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:38.371109 (+ 1305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:38.371117 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:38.371121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:38.371355 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:38.371995 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:38.372000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:38.372002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:38.372361 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:38.372530 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.373875 (+ 1345us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.374239 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":255,"threads_started":1}
I20260928 16:37:38.376102 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.347862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42873 (local address 127.10.247.195:38497)
0928 16:37:38.348879 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:38.348948 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:38.351314 (+ 2366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:38.352780 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:38.352798 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:38.353892 (+ 1094us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:38.355076 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.355106 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.356951 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.356956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.359365 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.359394 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.360518 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.360531 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.360980 (+ 449us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.362112 (+ 1132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:38.362145 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:38.365678 (+ 3533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:38.370001 (+ 4323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:38.370017 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:38.370038 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:38.370889 (+ 851us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:38.371528 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:38.371534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:38.371538 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:38.371842 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:38.372528 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:38.372538 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:38.373302 (+ 764us) client_negotiation.cc:769] Sending connection context
0928 16:37:38.373737 (+ 435us) client_negotiation.cc:240] Negotiation successful
0928 16:37:38.374463 (+ 726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":405,"threads_started":1}
I20260928 16:37:38.401793 19230 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:41837
I20260928 16:37:38.401764 19365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:41837 every 8 connection(s)
I20260928 16:37:38.403748 19230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:38.406456 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.389800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:36099 (local address 127.10.247.254:42873)
0928 16:37:38.390050 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.390057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.390701 (+ 644us) server_negotiation.cc:407] Connection header received
0928 16:37:38.391007 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.391014 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.391135 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.391250 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:38.393334 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.394373 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.396580 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.397124 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.397255 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.399225 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:38.399230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:38.399234 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:38.399275 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:38.402804 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:38.403675 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:38.403681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:38.403685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:38.403877 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:38.404369 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:38.404375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:38.404379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:38.404890 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:38.405089 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.405607 (+ 518us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.405923 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:37:38.406507 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.389790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42873 (local address 127.10.247.195:36099)
0928 16:37:38.390570 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:38.390593 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:38.390799 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:38.391513 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:38.391519 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:38.392434 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:38.393122 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.393139 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.394554 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.394563 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.396427 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.396443 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.397343 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.397349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.397508 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.398156 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:38.398176 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:38.399010 (+ 834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:38.403033 (+ 4023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:38.403038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:38.403041 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:38.403530 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:38.404044 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:38.404049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:38.404052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:38.404230 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:38.405038 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:38.405044 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:38.405297 (+ 253us) client_negotiation.cc:769] Sending connection context
0928 16:37:38.405505 (+ 208us) client_negotiation.cc:240] Negotiation successful
0928 16:37:38.405962 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":388,"threads_started":1}
I20260928 16:37:38.406939 19366 heartbeater.cc:347] Connected to a master server at 127.10.247.254:42873
I20260928 16:37:38.407220 19366 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:38.407618 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 19230
I20260928 16:37:38.407814 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:37:38.408320 19366 heartbeater.cc:511] Master 127.10.247.254:42873 requested a full tablet report, sending...
I20260928 16:37:38.408478 11231 external_mini_cluster.cc:1428] Setting key 2e1f1cf90562497a534775d783a0afcc
I20260928 16:37:38.410218 18878 ts_manager.cc:194] Registered new tserver with Master: bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837)
I20260928 16:37:38.412071 18878 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:38497
I20260928 16:37:38.412647 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 16:37:38 dist-test-slave-9fn6 krb5kdc[18833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 16:37:38 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:38 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 16:37:38.474293 19254 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:38.462852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.10.247.254:42873)
0928 16:37:38.463088 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.463124 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.463226 (+ 102us) server_negotiation.cc:407] Connection header received
0928 16:37:38.463503 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.463508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.463589 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.463686 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:38.465223 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.466010 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.467293 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.467746 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.467866 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.473895 (+ 6029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51376: BlockingRecv error: recv got EOF from 127.0.0.1:51376 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 28 16:37:38 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:38.492352 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.477721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51392 (local address 127.10.247.254:42873)
0928 16:37:38.477975 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.477981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.478275 (+ 294us) server_negotiation.cc:407] Connection header received
0928 16:37:38.478474 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.478478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.478563 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.478675 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:38.479916 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.480840 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.482169 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.482670 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.482801 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.486931 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:38.486936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:38.486939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:38.486984 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:38.489628 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:38.490303 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:38.490308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:38.490311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:38.490450 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:38.490812 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:38.490816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:38.490818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:38.491138 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:38.491433 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.491735 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.491999 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:37:38.497206 18878 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:37:38.504990 18878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:37:38.533066 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.519550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:41837 (local address 127.0.0.1:54262)
0928 16:37:38.520414 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:38.520486 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:38.520741 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:38.522485 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:38.522496 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:38.522550 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:38.523097 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.523113 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.526063 (+ 2950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.526071 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.529223 (+ 3152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.529246 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.531704 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.531711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.531983 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.532060 (+ 77us) client_negotiation.cc:769] Sending connection context
0928 16:37:38.532203 (+ 143us) client_negotiation.cc:240] Negotiation successful
0928 16:37:38.532478 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"spinlock_wait_cycles":7552,"thread_start_us":494,"threads_started":1}
I20260928 16:37:38.534854 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.520680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54262 (local address 127.10.247.195:41837)
0928 16:37:38.521655 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.521667 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.521709 (+ 42us) server_negotiation.cc:407] Connection header received
0928 16:37:38.521839 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.521849 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.522198 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.523360 (+ 1162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:38.523819 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.525868 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.529532 (+ 3664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.531411 (+ 1879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.533828 (+ 2417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.534107 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.534199 (+ 92us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.534424 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":1009,"threads_started":1}
I20260928 16:37:38.539805 19296 tablet_service.cc:1467] Processing CreateTablet for tablet 81757788e16e495a888b9bf12c741008 (DEFAULT_TABLE table=test-table [id=0559bb23e8da48118714b11019581545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:38.541422 19296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81757788e16e495a888b9bf12c741008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:38.542361 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.520165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:35365 (local address 127.0.0.1:41050)
0928 16:37:38.533379 (+ 13214us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:38.533399 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:38.533531 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:38.534909 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:38.534915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:38.534948 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:38.535154 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.535165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.537313 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.537316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.539424 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.539438 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.541399 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.541407 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.541769 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.541808 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 16:37:38.541908 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 16:37:38.542057 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13119,"thread_start_us":15287,"threads_started":1}
I20260928 16:37:38.542502 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.521768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.10.247.193:35365)
0928 16:37:38.533169 (+ 11401us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.533180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.533832 (+ 652us) server_negotiation.cc:407] Connection header received
0928 16:37:38.533973 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.533983 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.534386 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.534642 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:38.535718 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.537165 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.539587 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.541212 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.541396 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.541583 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.541998 (+ 415us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.542180 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11143,"thread_start_us":11730,"threads_started":1}
I20260928 16:37:38.544180 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.519904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:35667 (local address 127.0.0.1:60824)
0928 16:37:38.521229 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:38.521254 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:38.521384 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:38.537414 (+ 16030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:38.537420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:38.537454 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:38.537906 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.537918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.539678 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.539683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.541684 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.541699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.543374 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.543381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.543553 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.543585 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 16:37:38.543684 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 16:37:38.543819 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1202,"mutex_wait_us":1,"thread_start_us":2061,"threads_started":1}
I20260928 16:37:38.544823 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.521483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60824 (local address 127.10.247.194:35667)
0928 16:37:38.536735 (+ 15252us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.536745 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.536780 (+ 35us) server_negotiation.cc:407] Connection header received
0928 16:37:38.536885 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.536893 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.537157 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.537418 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:38.538109 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.539551 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.541829 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.543156 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.543339 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.543498 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.543738 (+ 240us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.543928 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14996,"thread_start_us":14836,"threads_started":1}
I20260928 16:37:38.547091 19016 tablet_service.cc:1467] Processing CreateTablet for tablet 81757788e16e495a888b9bf12c741008 (DEFAULT_TABLE table=test-table [id=0559bb23e8da48118714b11019581545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:38.548907 19016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81757788e16e495a888b9bf12c741008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:38.548872 19156 tablet_service.cc:1467] Processing CreateTablet for tablet 81757788e16e495a888b9bf12c741008 (DEFAULT_TABLE table=test-table [id=0559bb23e8da48118714b11019581545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:38.550439 19156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81757788e16e495a888b9bf12c741008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:38.555689 19390 tablet_bootstrap.cc:492] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: Bootstrap starting.
I20260928 16:37:38.560076 19390 tablet_bootstrap.cc:654] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:38.561735 19390 log.cc:824] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:38.564312 19392 tablet_bootstrap.cc:492] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: Bootstrap starting.
I20260928 16:37:38.566193 19390 tablet_bootstrap.cc:492] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: No bootstrap required, opened a new log
I20260928 16:37:38.566447 19390 ts_tablet_manager.cc:1397] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260928 16:37:38.567725 19393 tablet_bootstrap.cc:492] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: Bootstrap starting.
I20260928 16:37:38.568527 19392 tablet_bootstrap.cc:654] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:38.569787 19392 log.cc:824] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:38.571560 19393 tablet_bootstrap.cc:654] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:38.572918 19392 tablet_bootstrap.cc:492] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: No bootstrap required, opened a new log
I20260928 16:37:38.572933 19393 log.cc:824] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:38.573223 19392 ts_tablet_manager.cc:1397] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260928 16:37:38.577210 19393 tablet_bootstrap.cc:492] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: No bootstrap required, opened a new log
I20260928 16:37:38.577524 19393 ts_tablet_manager.cc:1397] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260928 16:37:38.578914 19390 raft_consensus.cc:348] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } }
I20260928 16:37:38.579480 19390 raft_consensus.cc:374] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:38.579588 19390 raft_consensus.cc:729] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde17e6deaee4f4bbf5f2daf4ea827f4, State: Initialized, Role: FOLLOWER
I20260928 16:37:38.580338 19390 consensus_queue.cc:260] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } }
I20260928 16:37:38.582000 19366 heartbeater.cc:503] Master 127.10.247.254:42873 was elected leader, sending a full tablet report...
I20260928 16:37:38.582798 19390 ts_tablet_manager.cc:1428] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260928 16:37:38.584448 19392 raft_consensus.cc:348] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } }
I20260928 16:37:38.585008 19392 raft_consensus.cc:374] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:38.585120 19392 raft_consensus.cc:729] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b94d34437b43869003e8f777246eb7, State: Initialized, Role: FOLLOWER
I20260928 16:37:38.585846 19392 consensus_queue.cc:260] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } }
I20260928 16:37:38.587203 19393 raft_consensus.cc:348] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } }
I20260928 16:37:38.587710 19393 raft_consensus.cc:374] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:38.587827 19393 raft_consensus.cc:729] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Initialized, Role: FOLLOWER
I20260928 16:37:38.588553 19393 consensus_queue.cc:260] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } }
I20260928 16:37:38.590065 19392 ts_tablet_manager.cc:1428] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260928 16:37:38.590826 19226 heartbeater.cc:503] Master 127.10.247.254:42873 was elected leader, sending a full tablet report...
I20260928 16:37:38.591624 19393 ts_tablet_manager.cc:1428] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260928 16:37:38.660985 19398 raft_consensus.cc:482] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260928 16:37:38.661551 19087 tablet.cc:2369] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:37:38.661494 19398 raft_consensus.cc:504] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } }
W20260928 16:37:38.662191 19367 tablet.cc:2369] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:37:38.663878 19398 leader_election.cc:288] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837), 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365)
I20260928 16:37:38.670953 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.664290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:54867 (local address 127.10.247.195:41837)
0928 16:37:38.664543 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.664550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.665273 (+ 723us) server_negotiation.cc:407] Connection header received
0928 16:37:38.665508 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.665512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.665583 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.665660 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:38.666257 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.667036 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.669122 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.669957 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.670129 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.670216 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.670519 (+ 303us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.670665 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 16:37:38.671037 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.664810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:43033 (local address 127.10.247.193:35365)
0928 16:37:38.665023 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.665029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.665361 (+ 332us) server_negotiation.cc:407] Connection header received
0928 16:37:38.665540 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.665546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.665634 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.665713 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:37:38.666379 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.667127 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.669204 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.670088 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.670250 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.670331 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.670627 (+ 296us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.670772 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 16:37:38.671098 19399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.664190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:41837 (local address 127.10.247.194:54867)
0928 16:37:38.665141 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:38.665162 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:38.665356 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:38.665688 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:38.665693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:38.665716 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:38.666111 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.666125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.667163 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.667169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.668974 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.668988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.670173 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.670179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.670363 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.670390 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 16:37:38.670478 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 16:37:38.670625 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":20,"thread_start_us":825,"threads_started":1}
I20260928 16:37:38.671617 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.664331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:35365 (local address 127.10.247.194:43033)
0928 16:37:38.665245 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:38.665259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:38.665397 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:38.665744 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:38.665747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:38.665770 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:37:38.666229 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.666238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.667273 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.667277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.668997 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.669012 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.670317 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.670323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.670480 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.670504 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 16:37:38.670584 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 16:37:38.670714 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":48,"thread_start_us":486,"threads_started":1}
I20260928 16:37:38.672204 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81757788e16e495a888b9bf12c741008" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" is_pre_election: true
I20260928 16:37:38.673033 19316 raft_consensus.cc:2454] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:38.672959 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81757788e16e495a888b9bf12c741008" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7" is_pre_election: true
I20260928 16:37:38.673568 19036 raft_consensus.cc:2454] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:38.673875 19109 leader_election.cc:302] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bde17e6deaee4f4bbf5f2daf4ea827f4, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:38.674439 19398 raft_consensus.cc:2781] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:38.674597 19398 raft_consensus.cc:482] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:38.674700 19398 raft_consensus.cc:3037] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:38.677274 19398 raft_consensus.cc:504] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } }
I20260928 16:37:38.677994 19398 leader_election.cc:288] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Requested vote from peers bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837), 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365)
I20260928 16:37:38.678550 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81757788e16e495a888b9bf12c741008" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7"
I20260928 16:37:38.678567 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81757788e16e495a888b9bf12c741008" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4"
I20260928 16:37:38.678803 19316 raft_consensus.cc:3037] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:38.678795 19036 raft_consensus.cc:3037] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:38.681636 19036 raft_consensus.cc:2454] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:38.681815 19316 raft_consensus.cc:2454] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:38.682132 19106 leader_election.cc:302] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b94d34437b43869003e8f777246eb7, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:38.682494 19398 raft_consensus.cc:2781] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:38.682997 19398 raft_consensus.cc:686] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 1 LEADER]: Becoming Leader. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Running, Role: LEADER
I20260928 16:37:38.683598 19398 consensus_queue.cc:237] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } }
I20260928 16:37:38.689577 18878 catalog_manager.cc:5495] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a reported cstate change: term changed from 0 to 1, leader changed from <none> to e328164a9271443eb58fc5eba50f268a (127.10.247.194). New cstate: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } health_report { overall_health: UNKNOWN } } }
I20260928 16:37:38.731340 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.725550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60838 (local address 127.10.247.194:35667)
0928 16:37:38.725780 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.725786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.725810 (+ 24us) server_negotiation.cc:407] Connection header received
0928 16:37:38.725935 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.725940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.726052 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.726208 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:37:38.726624 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.727397 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.728893 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.729328 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.729443 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.729955 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:37:38.730709 (+ 754us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:37:38.730879 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.730929 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.731030 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 16:37:38.741282 19316 raft_consensus.cc:1261] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Refusing update from remote peer e328164a9271443eb58fc5eba50f268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:38.741287 19036 raft_consensus.cc:1261] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Refusing update from remote peer e328164a9271443eb58fc5eba50f268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:38.742344 19398 consensus_queue.cc:1035] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:38.742717 19401 consensus_queue.cc:1035] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:38.777745 18877 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:38.778173 18877 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:38.783244 18877 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033: Sending DeleteTablet for 3 replicas of tablet 81757788e16e495a888b9bf12c741008
I20260928 16:37:38.785817 19296 tablet_service.cc:1514] Processing DeleteTablet for tablet 81757788e16e495a888b9bf12c741008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:38 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:54262
I20260928 16:37:38.786644 19016 tablet_service.cc:1514] Processing DeleteTablet for tablet 81757788e16e495a888b9bf12c741008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:38 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:41050
I20260928 16:37:38.786823 19414 tablet_replica.cc:330] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: stopping tablet replica
I20260928 16:37:38.787300 19414 raft_consensus.cc:2229] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:38.787710 19156 tablet_service.cc:1514] Processing DeleteTablet for tablet 81757788e16e495a888b9bf12c741008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:38 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:60824
I20260928 16:37:38.787848 19414 raft_consensus.cc:2258] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:38.788704 19415 tablet_replica.cc:330] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: stopping tablet replica
I20260928 16:37:38.789119 19415 raft_consensus.cc:2229] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:37:38.789779 19415 raft_consensus.cc:2258] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:38.791385 18877 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 16:37:38.791909 19415 ts_tablet_manager.cc:1905] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 16:37:38.792272 18877 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:37:38.795701 19416 tablet_replica.cc:330] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: stopping tablet replica
I20260928 16:37:38.796211 19416 raft_consensus.cc:2229] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:38.797751 19414 ts_tablet_manager.cc:1905] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:38.800871 19416 raft_consensus.cc:2258] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:38.801470 19296 tablet_service.cc:1467] Processing CreateTablet for tablet c779a023f8324c9691d65051f90aff7f (DEFAULT_TABLE table=test-table [id=1504a908ba8d4742a797fb8ad399b590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:38.801641 19415 ts_tablet_manager.cc:1918] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:38.801836 19415 log.cc:1196] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/81757788e16e495a888b9bf12c741008
I20260928 16:37:38.802435 19415 ts_tablet_manager.cc:1939] T 81757788e16e495a888b9bf12c741008 P e328164a9271443eb58fc5eba50f268a: Deleting consensus metadata
I20260928 16:37:38.803087 19156 tablet_service.cc:1467] Processing CreateTablet for tablet c779a023f8324c9691d65051f90aff7f (DEFAULT_TABLE table=test-table [id=1504a908ba8d4742a797fb8ad399b590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:38.804088 19016 tablet_service.cc:1467] Processing CreateTablet for tablet c779a023f8324c9691d65051f90aff7f (DEFAULT_TABLE table=test-table [id=1504a908ba8d4742a797fb8ad399b590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:38.804569 19016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c779a023f8324c9691d65051f90aff7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:38.805078 19414 ts_tablet_manager.cc:1918] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:38.805253 19414 log.cc:1196] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/81757788e16e495a888b9bf12c741008
I20260928 16:37:38.805516 19416 ts_tablet_manager.cc:1905] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:38.805774 19414 ts_tablet_manager.cc:1939] T 81757788e16e495a888b9bf12c741008 P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting consensus metadata
I20260928 16:37:38.807895 18865 catalog_manager.cc:4841] TS bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837): tablet 81757788e16e495a888b9bf12c741008 (table test-table [id=0559bb23e8da48118714b11019581545]) successfully deleted
I20260928 16:37:38.808126 19296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c779a023f8324c9691d65051f90aff7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:38.811192 19416 ts_tablet_manager.cc:1918] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:38.811352 19416 log.cc:1196] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/81757788e16e495a888b9bf12c741008
I20260928 16:37:38.811749 19392 tablet_bootstrap.cc:492] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: Bootstrap starting.
I20260928 16:37:38.811959 19416 ts_tablet_manager.cc:1939] T 81757788e16e495a888b9bf12c741008 P 28b94d34437b43869003e8f777246eb7: Deleting consensus metadata
I20260928 16:37:38.812371 19156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c779a023f8324c9691d65051f90aff7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:38.813618 19392 tablet_bootstrap.cc:654] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:38.814702 19390 tablet_bootstrap.cc:492] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: Bootstrap starting.
I20260928 16:37:38.816468 19390 tablet_bootstrap.cc:654] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:38.817664 18862 catalog_manager.cc:4841] TS 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365): tablet 81757788e16e495a888b9bf12c741008 (table test-table [id=0559bb23e8da48118714b11019581545]) successfully deleted
I20260928 16:37:38.818498 18863 catalog_manager.cc:4841] TS e328164a9271443eb58fc5eba50f268a (127.10.247.194:35667): tablet 81757788e16e495a888b9bf12c741008 (table test-table [id=0559bb23e8da48118714b11019581545]) successfully deleted
I20260928 16:37:38.823740 19393 tablet_bootstrap.cc:492] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: Bootstrap starting.
I20260928 16:37:38.824898 19390 tablet_bootstrap.cc:492] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: No bootstrap required, opened a new log
I20260928 16:37:38.825145 19390 ts_tablet_manager.cc:1397] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260928 16:37:38.825815 19393 tablet_bootstrap.cc:654] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:38.826256 19390 raft_consensus.cc:348] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:38.826612 19390 raft_consensus.cc:374] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:38.826758 19390 raft_consensus.cc:729] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde17e6deaee4f4bbf5f2daf4ea827f4, State: Initialized, Role: FOLLOWER
I20260928 16:37:38.827126 19390 consensus_queue.cc:260] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:38.828693 19390 ts_tablet_manager.cc:1428] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 16:37:38.830447 19392 tablet_bootstrap.cc:492] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: No bootstrap required, opened a new log
I20260928 16:37:38.830696 19392 ts_tablet_manager.cc:1397] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.007s
I20260928 16:37:38.831693 19392 raft_consensus.cc:348] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:38.831990 19392 raft_consensus.cc:374] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:38.832154 19392 raft_consensus.cc:729] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b94d34437b43869003e8f777246eb7, State: Initialized, Role: FOLLOWER
I20260928 16:37:38.832510 19392 consensus_queue.cc:260] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:38.833236 19392 ts_tablet_manager.cc:1428] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 16:37:38.835222 19393 tablet_bootstrap.cc:492] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: No bootstrap required, opened a new log
I20260928 16:37:38.835436 19393 ts_tablet_manager.cc:1397] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260928 16:37:38.836436 19393 raft_consensus.cc:348] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:38.836737 19393 raft_consensus.cc:374] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:38.836850 19393 raft_consensus.cc:729] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Initialized, Role: FOLLOWER
I20260928 16:37:38.837193 19393 consensus_queue.cc:260] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:38.837927 19393 ts_tablet_manager.cc:1428] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260928 16:37:38.904657 19410 raft_consensus.cc:482] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:38.904953 19410 raft_consensus.cc:504] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:38.905918 19410 leader_election.cc:288] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365), bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837)
I20260928 16:37:38.906610 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c779a023f8324c9691d65051f90aff7f" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" is_pre_election: true
I20260928 16:37:38.906620 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c779a023f8324c9691d65051f90aff7f" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7" is_pre_election: true
I20260928 16:37:38.906917 19316 raft_consensus.cc:2454] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:38.906934 19036 raft_consensus.cc:2454] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:38.907421 19106 leader_election.cc:302] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b94d34437b43869003e8f777246eb7, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:38.907789 19410 raft_consensus.cc:2781] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:38.907917 19410 raft_consensus.cc:482] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:38.908015 19410 raft_consensus.cc:3037] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:38.910638 19410 raft_consensus.cc:504] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:38.911527 19410 leader_election.cc:288] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Requested vote from peers 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365), bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837)
I20260928 16:37:38.911926 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c779a023f8324c9691d65051f90aff7f" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4"
I20260928 16:37:38.911945 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c779a023f8324c9691d65051f90aff7f" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7"
I20260928 16:37:38.912171 19316 raft_consensus.cc:3037] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:38.912182 19036 raft_consensus.cc:3037] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:38.914764 19316 raft_consensus.cc:2454] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:38.915323 19109 leader_election.cc:302] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bde17e6deaee4f4bbf5f2daf4ea827f4, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:38.915473 19036 raft_consensus.cc:2454] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:38.916275 19410 raft_consensus.cc:2781] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:38.916479 19410 raft_consensus.cc:686] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 1 LEADER]: Becoming Leader. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Running, Role: LEADER
I20260928 16:37:38.916883 19410 consensus_queue.cc:237] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:38.923835 18876 catalog_manager.cc:5495] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a reported cstate change: term changed from 0 to 1, leader changed from <none> to e328164a9271443eb58fc5eba50f268a (127.10.247.194). New cstate: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } health_report { overall_health: UNKNOWN } } }
Sep 28 16:37:38 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613456, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.254@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:38.996996 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.979940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.10.247.254:42873)
0928 16:37:38.980270 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:38.980277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:38.980932 (+ 655us) server_negotiation.cc:407] Connection header received
0928 16:37:38.981143 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:38.981148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:38.981236 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:38.981345 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:38.982808 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.984080 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.985523 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.986188 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.986320 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.990352 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:38.990357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:38.990361 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:38.990397 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:38.993210 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:38.994274 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:38.994278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:38.994281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:38.994458 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:38.994942 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:38.994992 (+ 50us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:38.994994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:38.995353 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:38.995500 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:38.996133 (+ 633us) server_negotiation.cc:299] Negotiation successful
0928 16:37:38.996543 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":152,"server-negotiator.queue_time_us":141}
I20260928 16:37:38.997054 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:38.980038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:42873 (local address 127.0.0.1:51396)
0928 16:37:38.980804 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:38.980826 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:38.981004 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:38.981391 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:38.981395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:38.982365 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:38.982660 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.982670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.984221 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.984227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.985388 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:38.985401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:38.986365 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:38.986369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:38.986489 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:38.987236 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:38.987253 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:38.990127 (+ 2874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:38.993361 (+ 3234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:38.993383 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:38.993393 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:38.994102 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:38.994570 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:38.994575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:38.994579 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:38.994768 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:38.995512 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:38.995523 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:38.995765 (+ 242us) client_negotiation.cc:769] Sending connection context
0928 16:37:38.995964 (+ 199us) client_negotiation.cc:240] Negotiation successful
0928 16:37:38.996357 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":393,"threads_started":1}
I20260928 16:37:39.004287 18876 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:51396:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 16:37:39.006029 18876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:37:39.014837 19016 tablet_service.cc:1467] Processing CreateTablet for tablet 46b861d5dbfa42b58e84358eb980dde7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ededfee1ea8406b96495536bcc966e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 16:37:39.014997 19156 tablet_service.cc:1467] Processing CreateTablet for tablet 46b861d5dbfa42b58e84358eb980dde7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ededfee1ea8406b96495536bcc966e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 16:37:39.015668 19016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46b861d5dbfa42b58e84358eb980dde7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:39.015677 19156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46b861d5dbfa42b58e84358eb980dde7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:39.017280 19296 tablet_service.cc:1467] Processing CreateTablet for tablet 46b861d5dbfa42b58e84358eb980dde7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ededfee1ea8406b96495536bcc966e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 16:37:39.017938 19296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46b861d5dbfa42b58e84358eb980dde7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:39.023941 19393 tablet_bootstrap.cc:492] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: Bootstrap starting.
I20260928 16:37:39.025902 19390 tablet_bootstrap.cc:492] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: Bootstrap starting.
I20260928 16:37:39.025960 19393 tablet_bootstrap.cc:654] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:39.027387 19392 tablet_bootstrap.cc:492] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: Bootstrap starting.
I20260928 16:37:39.028087 19390 tablet_bootstrap.cc:654] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:39.029798 19392 tablet_bootstrap.cc:654] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: Neither blocks nor log segments found. Creating new log.
W20260928 16:37:39.030072 19227 tablet.cc:2369] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:37:39.031566 19390 tablet_bootstrap.cc:492] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: No bootstrap required, opened a new log
I20260928 16:37:39.031801 19390 ts_tablet_manager.cc:1397] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260928 16:37:39.033128 19392 tablet_bootstrap.cc:492] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: No bootstrap required, opened a new log
I20260928 16:37:39.032920 19390 raft_consensus.cc:348] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.033268 19390 raft_consensus.cc:374] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:39.033366 19390 raft_consensus.cc:729] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde17e6deaee4f4bbf5f2daf4ea827f4, State: Initialized, Role: FOLLOWER
I20260928 16:37:39.033511 19392 ts_tablet_manager.cc:1397] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260928 16:37:39.033701 19390 consensus_queue.cc:260] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.034416 19390 ts_tablet_manager.cc:1428] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260928 16:37:39.034297 19396 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.034701 19396 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:37:39.034418 19392 raft_consensus.cc:348] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.034929 19392 raft_consensus.cc:374] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:39.035048 19392 raft_consensus.cc:729] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b94d34437b43869003e8f777246eb7, State: Initialized, Role: FOLLOWER
I20260928 16:37:39.035413 19392 consensus_queue.cc:260] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.035965 19397 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.036334 19397 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:37:39.036903 19392 ts_tablet_manager.cc:1428] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 16:37:39.043355 19393 tablet_bootstrap.cc:492] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: No bootstrap required, opened a new log
I20260928 16:37:39.043591 19393 ts_tablet_manager.cc:1397] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.000s
I20260928 16:37:39.044456 19393 raft_consensus.cc:348] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.044802 19393 raft_consensus.cc:374] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:39.044916 19393 raft_consensus.cc:729] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Initialized, Role: FOLLOWER
I20260928 16:37:39.045279 19393 consensus_queue.cc:260] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.047395 19410 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.047798 19410 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:37:39.048414 19393 ts_tablet_manager.cc:1428] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260928 16:37:39.231295 19410 raft_consensus.cc:482] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:39.231606 19410 raft_consensus.cc:504] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.232570 19410 leader_election.cc:288] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365), bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837)
I20260928 16:37:39.233264 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46b861d5dbfa42b58e84358eb980dde7" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" is_pre_election: true
I20260928 16:37:39.233335 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46b861d5dbfa42b58e84358eb980dde7" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7" is_pre_election: true
I20260928 16:37:39.233582 19316 raft_consensus.cc:2454] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:39.233680 19036 raft_consensus.cc:2454] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:39.234157 19109 leader_election.cc:302] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bde17e6deaee4f4bbf5f2daf4ea827f4, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:39.234503 19410 raft_consensus.cc:2781] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:39.234613 19410 raft_consensus.cc:482] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:39.234741 19410 raft_consensus.cc:3037] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:39.237403 19410 raft_consensus.cc:504] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.238143 19410 leader_election.cc:288] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Requested vote from peers 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365), bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837)
I20260928 16:37:39.238723 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46b861d5dbfa42b58e84358eb980dde7" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7"
I20260928 16:37:39.238812 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46b861d5dbfa42b58e84358eb980dde7" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4"
I20260928 16:37:39.239039 19036 raft_consensus.cc:3037] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:39.239076 19316 raft_consensus.cc:3037] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:39.242321 19036 raft_consensus.cc:2454] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:39.242321 19316 raft_consensus.cc:2454] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:39.242940 19106 leader_election.cc:302] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b94d34437b43869003e8f777246eb7, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:39.243304 19410 raft_consensus.cc:2781] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:39.243505 19410 raft_consensus.cc:686] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [term 1 LEADER]: Becoming Leader. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Running, Role: LEADER
I20260928 16:37:39.243841 19410 consensus_queue.cc:237] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.245816 19401 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: TxnStatusTablet state changed. Reason: New leader e328164a9271443eb58fc5eba50f268a. Latest consensus state: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.246140 19401 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: This TxnStatusTablet replica's current role is: LEADER
I20260928 16:37:39.246939 19431 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 16:37:39.247220 19431 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 16:37:39.247937 18877 catalog_manager.cc:5495] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a reported cstate change: term changed from 0 to 1, leader changed from <none> to e328164a9271443eb58fc5eba50f268a (127.10.247.194). New cstate: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } health_report { overall_health: UNKNOWN } } }
I20260928 16:37:39.289520 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:39.283902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:35667 (local address 127.0.0.1:60840)
0928 16:37:39.284242 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:39.284268 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:39.284448 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:39.284952 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:39.284957 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:39.284970 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 16:37:39.285193 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:39.285202 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.286235 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.286239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:39.287331 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:39.287344 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.288139 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.288145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:39.288370 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:39.288440 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 16:37:39.288897 (+ 457us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 16:37:39.288923 (+ 26us) client_negotiation.cc:769] Sending connection context
0928 16:37:39.288999 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 16:37:39.289188 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260928 16:37:39.289592 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:39.283945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60840 (local address 127.10.247.194:35667)
0928 16:37:39.284561 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:39.284568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:39.284589 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:37:39.284714 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:39.284720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:39.284804 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:39.284895 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:37:39.285336 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.286100 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.287477 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.287977 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.288092 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:39.288666 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:37:39.288801 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:37:39.288879 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:39.289054 (+ 175us) server_negotiation.cc:299] Negotiation successful
0928 16:37:39.289149 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":302,"threads_started":1}
I20260928 16:37:39.294718 19316 raft_consensus.cc:1261] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Refusing update from remote peer e328164a9271443eb58fc5eba50f268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:39.294808 19036 raft_consensus.cc:1261] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Refusing update from remote peer e328164a9271443eb58fc5eba50f268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:39.295444 19401 consensus_queue.cc:1035] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:39.295787 19410 consensus_queue.cc:1035] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:39.303459 19397 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: TxnStatusTablet state changed. Reason: New leader e328164a9271443eb58fc5eba50f268a. Latest consensus state: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.303818 19397 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:37:39.307109 19434 mvcc.cc:205] Tried to move back new op lower bound from 7334352729263665152 to 7334352729065992192. Current Snapshot: MvccSnapshot[applied={T|T < 7334352729263665152}]
I20260928 16:37:39.307843 19410 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.308250 19410 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: This TxnStatusTablet replica's current role is: LEADER
I20260928 16:37:39.313316 19396 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: TxnStatusTablet state changed. Reason: New leader e328164a9271443eb58fc5eba50f268a. Latest consensus state: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.313699 19396 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:37:39.315213 19410 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.315578 19410 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P e328164a9271443eb58fc5eba50f268a: This TxnStatusTablet replica's current role is: LEADER
I20260928 16:37:39.324920 19396 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.325212 19396 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P bde17e6deaee4f4bbf5f2daf4ea827f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:37:39.329423 19397 tablet_replica.cc:439] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } } }
I20260928 16:37:39.329839 19397 tablet_replica.cc:442] T 46b861d5dbfa42b58e84358eb980dde7 P 28b94d34437b43869003e8f777246eb7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 16:37:39.343547 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:39.336726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:53753 (local address 127.10.247.194:35667)
0928 16:37:39.336964 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:39.336974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:39.337605 (+ 631us) server_negotiation.cc:407] Connection header received
0928 16:37:39.337808 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:39.337814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:39.337919 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:39.338026 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:37:39.338585 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.339823 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.341367 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.342065 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.342203 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:39.342668 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:37:39.342828 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:37:39.342923 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:39.343096 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 16:37:39.343192 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":57}
I20260928 16:37:39.343572 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:39.336726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:35667 (local address 127.10.247.194:53753)
0928 16:37:39.337469 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:39.337487 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:39.337668 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:39.338031 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:39.338034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:39.338043 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 16:37:39.338422 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:39.338431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.339968 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.339972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:39.341221 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:39.341234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.342265 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.342269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:39.342403 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:39.342475 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 16:37:39.342951 (+ 476us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 16:37:39.342964 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 16:37:39.343055 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 16:37:39.343179 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":3,"thread_start_us":370,"threads_started":1}
I20260928 16:37:39.354346 19410 consensus_queue.cc:1035] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:39.354529 19035 raft_consensus.cc:1261] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Refusing update from remote peer e328164a9271443eb58fc5eba50f268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:39.355307 19410 consensus_queue.cc:1035] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:39.363965 19436 mvcc.cc:205] Tried to move back new op lower bound from 7334352729510256640 to 7334352727727771648. Current Snapshot: MvccSnapshot[applied={T|T < 7334352729510256640}]
I20260928 16:37:39.369910 19435 mvcc.cc:205] Tried to move back new op lower bound from 7334352729510256640 to 7334352727727771648. Current Snapshot: MvccSnapshot[applied={T|T < 7334352729510256640}]
I20260928 16:37:39.450592 18877 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:39.450870 18877 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:39.454494 18877 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033: Sending DeleteTablet for 3 replicas of tablet c779a023f8324c9691d65051f90aff7f
I20260928 16:37:39.455405 19016 tablet_service.cc:1514] Processing DeleteTablet for tablet c779a023f8324c9691d65051f90aff7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:41050
I20260928 16:37:39.455655 19296 tablet_service.cc:1514] Processing DeleteTablet for tablet c779a023f8324c9691d65051f90aff7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:54262
I20260928 16:37:39.456817 19156 tablet_service.cc:1514] Processing DeleteTablet for tablet c779a023f8324c9691d65051f90aff7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:60824
I20260928 16:37:39.465193 19453 tablet_replica.cc:330] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: stopping tablet replica
I20260928 16:37:39.465472 19453 raft_consensus.cc:2229] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:37:39.466024 19453 raft_consensus.cc:2258] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:39.467483 19453 ts_tablet_manager.cc:1905] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:39.472874 19452 tablet_replica.cc:330] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: stopping tablet replica
I20260928 16:37:39.473112 19452 raft_consensus.cc:2229] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:39.473390 19452 raft_consensus.cc:2258] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:39.474577 19452 ts_tablet_manager.cc:1905] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:39.476519 19453 ts_tablet_manager.cc:1918] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:37:39.476722 19453 log.cc:1196] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/c779a023f8324c9691d65051f90aff7f
I20260928 16:37:39.477283 19453 ts_tablet_manager.cc:1939] T c779a023f8324c9691d65051f90aff7f P e328164a9271443eb58fc5eba50f268a: Deleting consensus metadata
I20260928 16:37:39.478752 18863 catalog_manager.cc:4841] TS e328164a9271443eb58fc5eba50f268a (127.10.247.194:35667): tablet c779a023f8324c9691d65051f90aff7f (table test-table [id=1504a908ba8d4742a797fb8ad399b590]) successfully deleted
I20260928 16:37:39.481233 19451 tablet_replica.cc:330] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: stopping tablet replica
I20260928 16:37:39.481564 19451 raft_consensus.cc:2229] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:39.481938 19451 raft_consensus.cc:2258] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:39.483184 19451 ts_tablet_manager.cc:1905] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:39.485311 19452 ts_tablet_manager.cc:1918] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:37:39.485458 19452 log.cc:1196] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/c779a023f8324c9691d65051f90aff7f
I20260928 16:37:39.485849 19452 ts_tablet_manager.cc:1939] T c779a023f8324c9691d65051f90aff7f P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting consensus metadata
I20260928 16:37:39.487424 18865 catalog_manager.cc:4841] TS bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837): tablet c779a023f8324c9691d65051f90aff7f (table test-table [id=1504a908ba8d4742a797fb8ad399b590]) successfully deleted
I20260928 16:37:39.491401 19451 ts_tablet_manager.cc:1918] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:37:39.491556 19451 log.cc:1196] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/c779a023f8324c9691d65051f90aff7f
I20260928 16:37:39.492053 19451 ts_tablet_manager.cc:1939] T c779a023f8324c9691d65051f90aff7f P 28b94d34437b43869003e8f777246eb7: Deleting consensus metadata
I20260928 16:37:39.493580 18862 catalog_manager.cc:4841] TS 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365): tablet c779a023f8324c9691d65051f90aff7f (table test-table [id=1504a908ba8d4742a797fb8ad399b590]) successfully deleted
Sep 28 16:37:39 dist-test-slave-9fn6 krb5kdc[18833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 16:37:39 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:39 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 16:37:39.541076 19461 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:39.527315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.10.247.254:42873)
0928 16:37:39.528156 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:39.528163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:39.528188 (+ 25us) server_negotiation.cc:407] Connection header received
0928 16:37:39.528271 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:39.528276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:39.528368 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:39.528474 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:39.530188 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.531244 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.533250 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.533691 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.533796 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:39.540179 (+ 6383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51408: BlockingRecv error: recv got EOF from 127.0.0.1:51408 (error 108)
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":507,"threads_started":1}
Sep 28 16:37:39 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM
I20260928 16:37:39.559638 19461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:39.543502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.10.247.254:42873)
0928 16:37:39.543754 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:39.543761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:39.544253 (+ 492us) server_negotiation.cc:407] Connection header received
0928 16:37:39.544461 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:39.544466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:39.544555 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:39.544673 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:39.545963 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.546770 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.548085 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.548550 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.548715 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:39.553525 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:39.553534 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:39.553538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:39.553590 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:39.556741 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:39.557670 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:39.557674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:39.557677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:39.557832 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:39.558276 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:39.558280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:39.558282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:39.558598 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:39.558823 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:39.559145 (+ 322us) server_negotiation.cc:299] Negotiation successful
0928 16:37:39.559345 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 16:37:39.564404 18877 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:37:39.565322 18877 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:37:39.573325 19016 tablet_service.cc:1467] Processing CreateTablet for tablet c5dde8dc94ac4a43a62aa373b17ae51a (DEFAULT_TABLE table=test-table [id=4e6ed351b95b44e89fb5c19dc8ebc946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:39.573596 19296 tablet_service.cc:1467] Processing CreateTablet for tablet c5dde8dc94ac4a43a62aa373b17ae51a (DEFAULT_TABLE table=test-table [id=4e6ed351b95b44e89fb5c19dc8ebc946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:39.573856 19016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5dde8dc94ac4a43a62aa373b17ae51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:39.574090 19296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5dde8dc94ac4a43a62aa373b17ae51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:39.577719 19156 tablet_service.cc:1467] Processing CreateTablet for tablet c5dde8dc94ac4a43a62aa373b17ae51a (DEFAULT_TABLE table=test-table [id=4e6ed351b95b44e89fb5c19dc8ebc946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:39.578233 19156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5dde8dc94ac4a43a62aa373b17ae51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:39.588927 19468 tablet_bootstrap.cc:492] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: Bootstrap starting.
I20260928 16:37:39.590008 19469 tablet_bootstrap.cc:492] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: Bootstrap starting.
I20260928 16:37:39.591084 19468 tablet_bootstrap.cc:654] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:39.592209 19469 tablet_bootstrap.cc:654] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:39.593195 19467 tablet_bootstrap.cc:492] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: Bootstrap starting.
I20260928 16:37:39.594293 19468 tablet_bootstrap.cc:492] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: No bootstrap required, opened a new log
I20260928 16:37:39.594457 19468 ts_tablet_manager.cc:1397] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260928 16:37:39.595108 19467 tablet_bootstrap.cc:654] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:39.595336 19468 raft_consensus.cc:348] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.595603 19468 raft_consensus.cc:374] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:39.595692 19468 raft_consensus.cc:729] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde17e6deaee4f4bbf5f2daf4ea827f4, State: Initialized, Role: FOLLOWER
I20260928 16:37:39.595983 19468 consensus_queue.cc:260] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.596638 19468 ts_tablet_manager.cc:1428] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 16:37:39.605123 19469 tablet_bootstrap.cc:492] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: No bootstrap required, opened a new log
I20260928 16:37:39.605348 19469 ts_tablet_manager.cc:1397] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.009s
I20260928 16:37:39.605607 19467 tablet_bootstrap.cc:492] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: No bootstrap required, opened a new log
I20260928 16:37:39.605774 19467 ts_tablet_manager.cc:1397] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260928 16:37:39.606364 19469 raft_consensus.cc:348] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.606685 19469 raft_consensus.cc:374] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:39.606518 19467 raft_consensus.cc:348] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.606787 19467 raft_consensus.cc:374] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:39.606796 19469 raft_consensus.cc:729] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Initialized, Role: FOLLOWER
I20260928 16:37:39.606892 19467 raft_consensus.cc:729] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b94d34437b43869003e8f777246eb7, State: Initialized, Role: FOLLOWER
I20260928 16:37:39.607158 19467 consensus_queue.cc:260] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.607100 19469 consensus_queue.cc:260] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.607728 19467 ts_tablet_manager.cc:1428] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 16:37:39.608639 19469 ts_tablet_manager.cc:1428] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 16:37:39.695080 19410 raft_consensus.cc:482] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:39.695436 19410 raft_consensus.cc:504] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.696434 19410 leader_election.cc:288] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365), bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837)
I20260928 16:37:39.697275 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5dde8dc94ac4a43a62aa373b17ae51a" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" is_pre_election: true
I20260928 16:37:39.697295 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5dde8dc94ac4a43a62aa373b17ae51a" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7" is_pre_election: true
I20260928 16:37:39.697618 19036 raft_consensus.cc:2454] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:39.697618 19316 raft_consensus.cc:2454] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:39.698261 19109 leader_election.cc:302] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 28b94d34437b43869003e8f777246eb7, bde17e6deaee4f4bbf5f2daf4ea827f4, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:39.698712 19410 raft_consensus.cc:2781] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:39.698881 19410 raft_consensus.cc:482] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:39.699010 19410 raft_consensus.cc:3037] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:39.702757 19410 raft_consensus.cc:504] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.703676 19410 leader_election.cc:288] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Requested vote from peers 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365), bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837)
I20260928 16:37:39.704341 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5dde8dc94ac4a43a62aa373b17ae51a" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7"
I20260928 16:37:39.704397 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5dde8dc94ac4a43a62aa373b17ae51a" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4"
I20260928 16:37:39.704648 19036 raft_consensus.cc:3037] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:39.704736 19316 raft_consensus.cc:3037] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:39.707142 19036 raft_consensus.cc:2454] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:39.707718 19106 leader_election.cc:302] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b94d34437b43869003e8f777246eb7, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:39.708128 19410 raft_consensus.cc:2781] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:39.708218 19316 raft_consensus.cc:2454] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:39.708447 19410 raft_consensus.cc:686] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 1 LEADER]: Becoming Leader. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Running, Role: LEADER
I20260928 16:37:39.708864 19410 consensus_queue.cc:237] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.712461 18878 catalog_manager.cc:5495] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a reported cstate change: term changed from 0 to 1, leader changed from <none> to e328164a9271443eb58fc5eba50f268a (127.10.247.194). New cstate: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } health_report { overall_health: UNKNOWN } } }
I20260928 16:37:39.752233 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:39.746664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60856 (local address 127.10.247.194:35667)
0928 16:37:39.746897 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:39.746903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:39.747036 (+ 133us) server_negotiation.cc:407] Connection header received
0928 16:37:39.747276 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:39.747281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:39.747362 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:39.747441 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:37:39.747859 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.748777 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.750236 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.750767 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.750890 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:39.751378 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:37:39.751514 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:37:39.751599 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:39.751752 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 16:37:39.751854 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 16:37:39.755822 19316 raft_consensus.cc:1261] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Refusing update from remote peer e328164a9271443eb58fc5eba50f268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:39.755859 19036 raft_consensus.cc:1261] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Refusing update from remote peer e328164a9271443eb58fc5eba50f268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:39.756636 19447 consensus_queue.cc:1035] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:39.757093 19410 consensus_queue.cc:1035] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:39.777456 18876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:39.777834 18876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:39.782042 18876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033: Sending DeleteTablet for 3 replicas of tablet c5dde8dc94ac4a43a62aa373b17ae51a
I20260928 16:37:39.782948 19016 tablet_service.cc:1514] Processing DeleteTablet for tablet c5dde8dc94ac4a43a62aa373b17ae51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:41050
I20260928 16:37:39.783181 19296 tablet_service.cc:1514] Processing DeleteTablet for tablet c5dde8dc94ac4a43a62aa373b17ae51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:54262
I20260928 16:37:39.783365 19156 tablet_service.cc:1514] Processing DeleteTablet for tablet c5dde8dc94ac4a43a62aa373b17ae51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:60824
I20260928 16:37:39.783581 19296 tablet_service.cc:1514] Processing DeleteTablet for tablet c5dde8dc94ac4a43a62aa373b17ae51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:54262
I20260928 16:37:39.783720 19451 tablet_replica.cc:330] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: stopping tablet replica
I20260928 16:37:39.783747 19453 tablet_replica.cc:330] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: stopping tablet replica
I20260928 16:37:39.783926 19451 raft_consensus.cc:2229] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:39.783962 19453 raft_consensus.cc:2229] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:37:39.784157 19451 raft_consensus.cc:2258] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:39.784325 19453 raft_consensus.cc:2258] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:39.784423 19452 tablet_replica.cc:330] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: stopping tablet replica
I20260928 16:37:39.784624 19452 raft_consensus.cc:2229] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:39.784878 19452 raft_consensus.cc:2258] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:39.785488 19453 ts_tablet_manager.cc:1905] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:39.786135 18876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:37:39.786911 18876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:37:39.791683 19453 ts_tablet_manager.cc:1918] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:39.791887 19453 log.cc:1196] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/c5dde8dc94ac4a43a62aa373b17ae51a
I20260928 16:37:39.792356 19453 ts_tablet_manager.cc:1939] T c5dde8dc94ac4a43a62aa373b17ae51a P e328164a9271443eb58fc5eba50f268a: Deleting consensus metadata
I20260928 16:37:39.793617 19451 ts_tablet_manager.cc:1905] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:39.794149 19452 ts_tablet_manager.cc:1905] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:39.797075 18863 catalog_manager.cc:4841] TS e328164a9271443eb58fc5eba50f268a (127.10.247.194:35667): tablet c5dde8dc94ac4a43a62aa373b17ae51a (table test-table [id=4e6ed351b95b44e89fb5c19dc8ebc946]) successfully deleted
I20260928 16:37:39.799048 19016 tablet_service.cc:1467] Processing CreateTablet for tablet 5057a2316dab4dd2b61b3c543d16aaab (DEFAULT_TABLE table=test-table [id=f3d7c82a7b3d436fa7d5a5d817ebc792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:39.799610 19016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5057a2316dab4dd2b61b3c543d16aaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:39.799623 19296 tablet_service.cc:1467] Processing CreateTablet for tablet 5057a2316dab4dd2b61b3c543d16aaab (DEFAULT_TABLE table=test-table [id=f3d7c82a7b3d436fa7d5a5d817ebc792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:39.800088 19296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5057a2316dab4dd2b61b3c543d16aaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:39.803395 19451 ts_tablet_manager.cc:1918] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:39.803613 19451 log.cc:1196] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/c5dde8dc94ac4a43a62aa373b17ae51a
I20260928 16:37:39.804116 19451 ts_tablet_manager.cc:1939] T c5dde8dc94ac4a43a62aa373b17ae51a P 28b94d34437b43869003e8f777246eb7: Deleting consensus metadata
I20260928 16:37:39.805137 19156 tablet_service.cc:1467] Processing CreateTablet for tablet 5057a2316dab4dd2b61b3c543d16aaab (DEFAULT_TABLE table=test-table [id=f3d7c82a7b3d436fa7d5a5d817ebc792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:37:39.805709 19156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5057a2316dab4dd2b61b3c543d16aaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:37:39.805660 18862 catalog_manager.cc:4841] TS 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365): tablet c5dde8dc94ac4a43a62aa373b17ae51a (table test-table [id=4e6ed351b95b44e89fb5c19dc8ebc946]) successfully deleted
I20260928 16:37:39.808279 19468 tablet_bootstrap.cc:492] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: Bootstrap starting.
I20260928 16:37:39.810148 19468 tablet_bootstrap.cc:654] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:39.812479 19469 tablet_bootstrap.cc:492] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: Bootstrap starting.
I20260928 16:37:39.814742 19469 tablet_bootstrap.cc:654] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:39.816864 19452 ts_tablet_manager.cc:1918] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:37:39.818292 19467 tablet_bootstrap.cc:492] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: Bootstrap starting.
I20260928 16:37:39.819437 19452 log.cc:1196] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/c5dde8dc94ac4a43a62aa373b17ae51a
I20260928 16:37:39.820062 19452 ts_tablet_manager.cc:1939] T c5dde8dc94ac4a43a62aa373b17ae51a P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting consensus metadata
I20260928 16:37:39.820397 19467 tablet_bootstrap.cc:654] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:39.822121 18865 catalog_manager.cc:4841] TS bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837): tablet c5dde8dc94ac4a43a62aa373b17ae51a (table test-table [id=4e6ed351b95b44e89fb5c19dc8ebc946]) successfully deleted
I20260928 16:37:39.822309 19468 tablet_bootstrap.cc:492] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: No bootstrap required, opened a new log
I20260928 16:37:39.822530 19468 ts_tablet_manager.cc:1397] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260928 16:37:39.823567 19468 raft_consensus.cc:348] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
W20260928 16:37:39.823685 18865 catalog_manager.cc:4808] TS bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837): delete failed for tablet c5dde8dc94ac4a43a62aa373b17ae51a because the tablet was not found. No further retry: Not found: Tablet not found: c5dde8dc94ac4a43a62aa373b17ae51a
I20260928 16:37:39.823886 19468 raft_consensus.cc:374] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:39.823963 19468 raft_consensus.cc:729] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde17e6deaee4f4bbf5f2daf4ea827f4, State: Initialized, Role: FOLLOWER
I20260928 16:37:39.824954 19468 consensus_queue.cc:260] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.826294 19469 tablet_bootstrap.cc:492] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: No bootstrap required, opened a new log
I20260928 16:37:39.826292 19467 tablet_bootstrap.cc:492] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: No bootstrap required, opened a new log
I20260928 16:37:39.826294 19468 ts_tablet_manager.cc:1428] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260928 16:37:39.826489 19467 ts_tablet_manager.cc:1397] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 16:37:39.826563 19469 ts_tablet_manager.cc:1397] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20260928 16:37:39.827280 19469 raft_consensus.cc:348] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.827502 19469 raft_consensus.cc:374] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:39.827375 19467 raft_consensus.cc:348] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.827582 19469 raft_consensus.cc:729] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Initialized, Role: FOLLOWER
I20260928 16:37:39.827672 19467 raft_consensus.cc:374] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:39.827770 19467 raft_consensus.cc:729] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b94d34437b43869003e8f777246eb7, State: Initialized, Role: FOLLOWER
I20260928 16:37:39.827841 19469 consensus_queue.cc:260] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.828053 19467 consensus_queue.cc:260] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.828485 19469 ts_tablet_manager.cc:1428] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260928 16:37:39.828572 19467 ts_tablet_manager.cc:1428] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 16:37:39.880304 19410 raft_consensus.cc:482] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:37:39.880575 19410 raft_consensus.cc:504] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.881527 19410 leader_election.cc:288] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365), bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837)
I20260928 16:37:39.882206 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5057a2316dab4dd2b61b3c543d16aaab" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" is_pre_election: true
I20260928 16:37:39.882239 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5057a2316dab4dd2b61b3c543d16aaab" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7" is_pre_election: true
I20260928 16:37:39.882615 19036 raft_consensus.cc:2454] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:39.882615 19316 raft_consensus.cc:2454] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 0.
I20260928 16:37:39.883157 19106 leader_election.cc:302] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 28b94d34437b43869003e8f777246eb7, bde17e6deaee4f4bbf5f2daf4ea827f4, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:39.883522 19410 raft_consensus.cc:2781] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:37:39.883646 19410 raft_consensus.cc:482] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:37:39.883736 19410 raft_consensus.cc:3037] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:39.886237 19410 raft_consensus.cc:504] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.886938 19410 leader_election.cc:288] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Requested vote from peers 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365), bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837)
I20260928 16:37:39.887579 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5057a2316dab4dd2b61b3c543d16aaab" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b94d34437b43869003e8f777246eb7"
I20260928 16:37:39.887584 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5057a2316dab4dd2b61b3c543d16aaab" candidate_uuid: "e328164a9271443eb58fc5eba50f268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4"
I20260928 16:37:39.887840 19036 raft_consensus.cc:3037] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:39.887844 19316 raft_consensus.cc:3037] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:39.890329 19036 raft_consensus.cc:2454] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:39.890388 19316 raft_consensus.cc:2454] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e328164a9271443eb58fc5eba50f268a in term 1.
I20260928 16:37:39.890903 19109 leader_election.cc:302] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 28b94d34437b43869003e8f777246eb7, bde17e6deaee4f4bbf5f2daf4ea827f4, e328164a9271443eb58fc5eba50f268a; no voters:
I20260928 16:37:39.891245 19410 raft_consensus.cc:2781] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:39.891420 19410 raft_consensus.cc:686] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 1 LEADER]: Becoming Leader. State: Replica: e328164a9271443eb58fc5eba50f268a, State: Running, Role: LEADER
I20260928 16:37:39.891760 19410 consensus_queue.cc:237] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } }
I20260928 16:37:39.894764 18878 catalog_manager.cc:5495] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a reported cstate change: term changed from 0 to 1, leader changed from <none> to e328164a9271443eb58fc5eba50f268a (127.10.247.194). New cstate: current_term: 1 leader_uuid: "e328164a9271443eb58fc5eba50f268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e328164a9271443eb58fc5eba50f268a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 35667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 } health_report { overall_health: UNKNOWN } } }
I20260928 16:37:39.920639 19315 raft_consensus.cc:1261] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Refusing update from remote peer e328164a9271443eb58fc5eba50f268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:39.920779 19036 raft_consensus.cc:1261] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Refusing update from remote peer e328164a9271443eb58fc5eba50f268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:37:39.921334 19401 consensus_queue.cc:1035] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b94d34437b43869003e8f777246eb7" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 35365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:39.921627 19410 consensus_queue.cc:1035] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bde17e6deaee4f4bbf5f2daf4ea827f4" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 41837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:37:39.969313 18878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:39.969600 18878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:37:39.973150 18878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 859e198a0a9c4586b149752c11f70033: Sending DeleteTablet for 3 replicas of tablet 5057a2316dab4dd2b61b3c543d16aaab
I20260928 16:37:39.974169 19016 tablet_service.cc:1514] Processing DeleteTablet for tablet 5057a2316dab4dd2b61b3c543d16aaab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:41050
I20260928 16:37:39.974237 19155 tablet_service.cc:1514] Processing DeleteTablet for tablet 5057a2316dab4dd2b61b3c543d16aaab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:60824
I20260928 16:37:39.974380 19296 tablet_service.cc:1514] Processing DeleteTablet for tablet 5057a2316dab4dd2b61b3c543d16aaab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:37:39 UTC) from {username='oryx', principal='oryx/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:54262
I20260928 16:37:39.974570 19451 tablet_replica.cc:330] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: stopping tablet replica
I20260928 16:37:39.974606 19453 tablet_replica.cc:330] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: stopping tablet replica
I20260928 16:37:39.974706 19452 tablet_replica.cc:330] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: stopping tablet replica
I20260928 16:37:39.974749 19451 raft_consensus.cc:2229] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:39.974808 19453 raft_consensus.cc:2229] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:37:39.974879 19452 raft_consensus.cc:2229] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:37:39.974947 19451 raft_consensus.cc:2258] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:39.975111 19452 raft_consensus.cc:2258] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:39.975232 19453 raft_consensus.cc:2258] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:37:39.976276 19452 ts_tablet_manager.cc:1905] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:39.976351 19453 ts_tablet_manager.cc:1905] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:39.976409 19451 ts_tablet_manager.cc:1905] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:37:39.976485 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 18950
I20260928 16:37:39.983942 19452 ts_tablet_manager.cc:1918] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:37:39.984124 19452 log.cc:1196] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/5057a2316dab4dd2b61b3c543d16aaab
I20260928 16:37:39.984347 19451 ts_tablet_manager.cc:1918] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:37:39.984591 19452 ts_tablet_manager.cc:1939] T 5057a2316dab4dd2b61b3c543d16aaab P bde17e6deaee4f4bbf5f2daf4ea827f4: Deleting consensus metadata
I20260928 16:37:39.984582 19451 log.cc:1196] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/5057a2316dab4dd2b61b3c543d16aaab
I20260928 16:37:39.985110 19451 ts_tablet_manager.cc:1939] T 5057a2316dab4dd2b61b3c543d16aaab P 28b94d34437b43869003e8f777246eb7: Deleting consensus metadata
I20260928 16:37:39.985116 19453 ts_tablet_manager.cc:1918] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 16:37:39.985284 19453 log.cc:1196] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/5057a2316dab4dd2b61b3c543d16aaab
I20260928 16:37:39.985739 19453 ts_tablet_manager.cc:1939] T 5057a2316dab4dd2b61b3c543d16aaab P e328164a9271443eb58fc5eba50f268a: Deleting consensus metadata
I20260928 16:37:39.986071 18865 catalog_manager.cc:4841] TS bde17e6deaee4f4bbf5f2daf4ea827f4 (127.10.247.195:41837): tablet 5057a2316dab4dd2b61b3c543d16aaab (table test-table [id=f3d7c82a7b3d436fa7d5a5d817ebc792]) successfully deleted
I20260928 16:37:39.986609 18862 catalog_manager.cc:4841] TS 28b94d34437b43869003e8f777246eb7 (127.10.247.193:35365): tablet 5057a2316dab4dd2b61b3c543d16aaab (table test-table [id=f3d7c82a7b3d436fa7d5a5d817ebc792]) successfully deleted
I20260928 16:37:39.987017 18863 catalog_manager.cc:4841] TS e328164a9271443eb58fc5eba50f268a (127.10.247.194:35667): tablet 5057a2316dab4dd2b61b3c543d16aaab (table test-table [id=f3d7c82a7b3d436fa7d5a5d817ebc792]) successfully deleted
Sep 28 16:37:39 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.193@KRBTEST.COM
I20260928 16:37:40.004132 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:39.981592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.10.247.193:35365)
0928 16:37:39.987187 (+ 5595us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:39.987194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:39.987707 (+ 513us) server_negotiation.cc:407] Connection header received
0928 16:37:39.987858 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:39.987862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:39.987944 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:39.988053 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:39.989459 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.990341 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.991750 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:39.992227 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:39.992328 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:39.997061 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:39.997086 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:39.997107 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:39.997167 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:40.000456 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:40.001553 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:40.001565 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:40.001567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:40.001742 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:40.002228 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:40.002233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:40.002236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:40.002786 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:40.003094 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:40.003464 (+ 370us) server_negotiation.cc:299] Negotiation successful
0928 16:37:40.003696 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5466,"thread_start_us":5607,"threads_started":1}
I20260928 16:37:40.004951 19081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:40.113214 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 18950
I20260928 16:37:40.132887 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 19090
Sep 28 16:37:40 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.194@KRBTEST.COM
I20260928 16:37:40.149585 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:40.133801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60866 (local address 127.10.247.194:35667)
0928 16:37:40.134037 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:40.134043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:40.134065 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:37:40.134280 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:40.134285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:40.134387 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:40.134504 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:40.135868 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:40.136769 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:40.138129 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:40.138577 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:40.138677 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:40.143387 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:40.143428 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:40.143449 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:40.143509 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:40.146741 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:40.147474 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:40.147482 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:40.147484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:40.147641 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:40.147978 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:40.147981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:40.147983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:40.148398 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:40.148737 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:40.149031 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 16:37:40.149231 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260928 16:37:40.150416 19221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:40.320914 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 19090
I20260928 16:37:40.347652 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 19230
Sep 28 16:37:40 dist-test-slave-9fn6 krb5kdc[18833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.247.195@KRBTEST.COM
I20260928 16:37:40.365265 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:40.348686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.10.247.195:41837)
0928 16:37:40.349252 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:40.349258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:40.349278 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:37:40.349350 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:40.349354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:40.349428 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:40.349531 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:40.350967 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:40.351806 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:40.353251 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:40.353704 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:40.353809 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:40.358840 (+ 5031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:40.358861 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:40.358882 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:40.358936 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:40.362121 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:40.362875 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:40.362884 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:40.362886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:40.363045 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:40.363444 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:40.363448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:40.363450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:40.363920 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:40.364261 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:40.364633 (+ 372us) server_negotiation.cc:299] Negotiation successful
0928 16:37:40.364887 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":269,"threads_started":1}
I20260928 16:37:40.365983 19361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:40.471485 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 19230
I20260928 16:37:40.489822 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 18846
I20260928 16:37:40.491333 18919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:40.624971 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 18846
2026-09-28T16:37:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6063 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:37:40 dist-test-slave-9fn6 krb5kdc[19494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:37:40 dist-test-slave-9fn6 krb5kdc[19494](info): set up 2 sockets
Sep 28 16:37:40 dist-test-slave-9fn6 krb5kdc[19494](info): commencing operation
krb5kdc: starting...
W20260928 16:37:41.673739 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.005s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:37:41 dist-test-slave-9fn6 krb5kdc[19494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:37:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:37:41.944720 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:35871
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:35871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:42.330096 19507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:42.330482 19507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:42.330614 19507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:42.338869 19507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:42.339027 19507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:42.339093 19507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:42.339182 19507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:42.339242 19507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:42.350473 19507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:35871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:35871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:42.352311 19507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:42.354036 19507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:42.367619 19512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:42.367655 19515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:42.367620 19513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:42.368709 19507 server_base.cc:1034] running on GCE node
I20260928 16:37:42.369771 19507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:42.371608 19507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:42.372841 19507 hybrid_clock.cc:648] HybridClock initialized: now 1790613462372776 us; error 55 us; skew 500 ppm
I20260928 16:37:42.373289 19507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-28T16:37:42Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1824 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:37:42 dist-test-slave-9fn6 krb5kdc[19521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:37:42 dist-test-slave-9fn6 krb5kdc[19521](info): set up 2 sockets
Sep 28 16:37:42 dist-test-slave-9fn6 krb5kdc[19521](info): commencing operation
krb5kdc: starting...
W20260928 16:37:43.493912 11231 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:37:43 dist-test-slave-9fn6 krb5kdc[19521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:37:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:37:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:37:43.762794 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38351
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:44.148012 19535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:44.148458 19535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:44.148587 19535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:44.156551 19535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:44.156749 19535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:44.156813 19535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:44.156873 19535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:44.156919 19535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:44.167883 19535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:38351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:44.169770 19535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:44.171418 19535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:44.185242 19540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:44.185333 19543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:44.185261 19541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:44.186244 19535 server_base.cc:1034] running on GCE node
I20260928 16:37:44.187249 19535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:44.189148 19535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:44.190410 19535 hybrid_clock.cc:648] HybridClock initialized: now 1790613464190335 us; error 62 us; skew 500 ppm
I20260928 16:37:44.190833 19535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:44 dist-test-slave-9fn6 krb5kdc[19521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613464, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:44.194677 19535 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:37:44.196555 19535 webserver.cc:460] Webserver started at http://127.10.247.254:40785/ using document root <none> and password file <none>
I20260928 16:37:44.197515 19535 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:44.197651 19535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:44.198020 19535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:44.201974 19535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "42294aed648147e280ea020cb8e4a3ab"
format_stamp: "Formatted at 2026-09-28 16:37:44 on dist-test-slave-9fn6"
server_key: "f6c3b889149a784e35ea2f79ef97d38b"
server_key_iv: "5f6e1413cde7e9a3cb9a936343b2f8df"
server_key_version: "encryptionkey@0"
I20260928 16:37:44.202889 19535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "42294aed648147e280ea020cb8e4a3ab"
format_stamp: "Formatted at 2026-09-28 16:37:44 on dist-test-slave-9fn6"
server_key: "f6c3b889149a784e35ea2f79ef97d38b"
server_key_iv: "5f6e1413cde7e9a3cb9a936343b2f8df"
server_key_version: "encryptionkey@0"
I20260928 16:37:44.209408 19535 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260928 16:37:44.213845 19550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:44.215688 19535 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:37:44.215909 19535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "42294aed648147e280ea020cb8e4a3ab"
format_stamp: "Formatted at 2026-09-28 16:37:44 on dist-test-slave-9fn6"
server_key: "f6c3b889149a784e35ea2f79ef97d38b"
server_key_iv: "5f6e1413cde7e9a3cb9a936343b2f8df"
server_key_version: "encryptionkey@0"
I20260928 16:37:44.216142 19535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:44.231827 19535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:44.237272 19535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:44.237622 19535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:44.258352 19535 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:38351
I20260928 16:37:44.258356 19601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:38351 every 8 connection(s)
I20260928 16:37:44.260071 19535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:44.261289 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 19535
I20260928 16:37:44.261564 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:37:44.262457 11231 external_mini_cluster.cc:1428] Setting key dce992a33eb052641fc00553c5bdf9a1
I20260928 16:37:44.265798 19602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 16:37:44 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:44.282922 19602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: Bootstrap starting.
I20260928 16:37:44.287832 19602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: Neither blocks nor log segments found. Creating new log.
I20260928 16:37:44.289469 19602 log.cc:824] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:44.291728 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:44.264904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36752 (local address 127.10.247.254:38351)
0928 16:37:44.265756 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:44.265769 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:44.265821 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:37:44.267724 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:44.267750 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:44.268204 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:44.268592 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:44.270650 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:44.272292 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:44.274364 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:44.275435 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:44.275664 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:44.281865 (+ 6201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:44.281878 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:44.281903 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:44.281973 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:44.285775 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:44.287032 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:44.287043 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:44.287052 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:44.287272 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:44.287857 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:44.287862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:44.287865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:44.288770 (+ 905us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:44.289078 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:44.289687 (+ 609us) server_negotiation.cc:299] Negotiation successful
0928 16:37:44.290277 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":369,"threads_started":1}
I20260928 16:37:44.293672 19602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: No bootstrap required, opened a new log
I20260928 16:37:44.302721 19602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } }
I20260928 16:37:44.303236 19602 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:37:44.303323 19602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42294aed648147e280ea020cb8e4a3ab, State: Initialized, Role: FOLLOWER
I20260928 16:37:44.303993 19602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } }
I20260928 16:37:44.304252 19602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:44.304364 19602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:44.304529 19602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:37:44.307281 19602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } }
I20260928 16:37:44.307888 19602 leader_election.cc:302] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42294aed648147e280ea020cb8e4a3ab; no voters:
I20260928 16:37:44.308531 19602 leader_election.cc:288] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:37:44.308709 19607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:37:44.309412 19607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 1 LEADER]: Becoming Leader. State: Replica: 42294aed648147e280ea020cb8e4a3ab, State: Running, Role: LEADER
I20260928 16:37:44.310097 19607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } }
I20260928 16:37:44.310937 19602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:44.313031 19608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42294aed648147e280ea020cb8e4a3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } } }
I20260928 16:37:44.313380 19608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:44.316244 19609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42294aed648147e280ea020cb8e4a3ab. Latest consensus state: current_term: 1 leader_uuid: "42294aed648147e280ea020cb8e4a3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } } }
I20260928 16:37:44.317660 19609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:44.318504 19613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:44.326195 19613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:44.338469 19613 catalog_manager.cc:1347] Generated new cluster ID: 30faa1a0fed14f8cb04907d9604780c0
I20260928 16:37:44.338652 19613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:44.352041 19613 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:37:44.353704 19613 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:44.369917 19613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: Generated new TSK 0
I20260928 16:37:44.371143 19613 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:37:44.448736 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38351
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:44.849875 19629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:44.850263 19629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:44.850415 19629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:44.858429 19629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:44.858590 19629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:44.858755 19629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:37:44.869879 19629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:44.871737 19629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:44.873526 19629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:44.889096 19635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:44.889009 19637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:44.889055 19634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:44.894142 19629 server_base.cc:1034] running on GCE node
I20260928 16:37:44.894869 19629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:44.896030 19629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:44.897235 19629 hybrid_clock.cc:648] HybridClock initialized: now 1790613464897174 us; error 54 us; skew 500 ppm
I20260928 16:37:44.897595 19629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:44 dist-test-slave-9fn6 krb5kdc[19521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613464, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:44.901340 19629 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:37:44.903196 19629 webserver.cc:460] Webserver started at http://127.10.247.193:35249/ using document root <none> and password file <none>
I20260928 16:37:44.904054 19629 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:44.904174 19629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:44.904537 19629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:44.907994 19629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "eda316f456f346f19be27cf7a4bf7e87"
format_stamp: "Formatted at 2026-09-28 16:37:44 on dist-test-slave-9fn6"
server_key: "2658381825adb165bc12f458a2b7fef4"
server_key_iv: "567a43882a1b5aed209bd1829562dd09"
server_key_version: "encryptionkey@0"
I20260928 16:37:44.908927 19629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "eda316f456f346f19be27cf7a4bf7e87"
format_stamp: "Formatted at 2026-09-28 16:37:44 on dist-test-slave-9fn6"
server_key: "2658381825adb165bc12f458a2b7fef4"
server_key_iv: "567a43882a1b5aed209bd1829562dd09"
server_key_version: "encryptionkey@0"
I20260928 16:37:44.914929 19629 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:37:44.919068 19644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:44.920776 19629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:37:44.920956 19629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "eda316f456f346f19be27cf7a4bf7e87"
format_stamp: "Formatted at 2026-09-28 16:37:44 on dist-test-slave-9fn6"
server_key: "2658381825adb165bc12f458a2b7fef4"
server_key_iv: "567a43882a1b5aed209bd1829562dd09"
server_key_version: "encryptionkey@0"
I20260928 16:37:44.921216 19629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:44.942049 19629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:44.947288 19629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:44.947626 19629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:44.948880 19629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:44.950803 19629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:44.950897 19629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:44.951130 19629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:44.951233 19629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:44.986833 19629 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:41355
I20260928 16:37:44.986889 19756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:41355 every 8 connection(s)
I20260928 16:37:44.988430 19629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:44.993283 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 19629
I20260928 16:37:44.993528 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:37:44.994333 11231 external_mini_cluster.cc:1428] Setting key 0c7212320f879b4f9638de72889dd4de
Sep 28 16:37:45 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613464, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:45.014272 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:44.991546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:57011 (local address 127.10.247.254:38351)
0928 16:37:44.992107 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:44.992115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:44.993374 (+ 1259us) server_negotiation.cc:407] Connection header received
0928 16:37:44.995875 (+ 2501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:44.995881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:44.996002 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:44.996116 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:44.998440 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:44.999643 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:45.001676 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:45.002292 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:45.002435 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:45.007217 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:45.007224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:45.007227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:45.007276 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:45.010180 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:45.011240 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:45.011246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:45.011249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:45.011412 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:45.011980 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:45.011985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:45.011987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:45.012350 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:45.012511 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:45.013512 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0928 16:37:45.013817 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":254,"threads_started":1}
I20260928 16:37:45.015614 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:44.992170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:57011)
0928 16:37:44.993169 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:44.993218 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:44.995505 (+ 2287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:44.996506 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:44.996521 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:44.997414 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:44.998205 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:44.998227 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:44.999867 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:44.999872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:45.001504 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:45.001522 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:45.002594 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:45.002601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:45.002903 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:45.003793 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:45.003832 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:45.006842 (+ 3010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:45.010392 (+ 3550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:45.010406 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:45.010427 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:45.011072 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:45.011536 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:45.011541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:45.011544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:45.011847 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:45.012494 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:45.012505 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:45.013058 (+ 553us) client_negotiation.cc:769] Sending connection context
0928 16:37:45.013367 (+ 309us) client_negotiation.cc:240] Negotiation successful
0928 16:37:45.014086 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":444,"threads_started":1}
I20260928 16:37:45.017831 19757 heartbeater.cc:347] Connected to a master server at 127.10.247.254:38351
I20260928 16:37:45.018283 19757 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:45.019521 19757 heartbeater.cc:511] Master 127.10.247.254:38351 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
I20260928 16:37:45.022840 19567 ts_manager.cc:194] Registered new tserver with Master: eda316f456f346f19be27cf7a4bf7e87 (127.10.247.193:41355)
I20260928 16:37:45.024654 19567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:57011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:37:45.056988 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38351
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:45.447551 19764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:45.447984 19764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:45.448125 19764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:45.456476 19764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:45.456689 19764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:45.456871 19764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:45.468211 19764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:45.470036 19764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:45.471784 19764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:45.487536 19769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:45.487573 19772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:45.487624 19770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:45.488373 19764 server_base.cc:1034] running on GCE node
I20260928 16:37:45.489154 19764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:45.490362 19764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:45.491600 19764 hybrid_clock.cc:648] HybridClock initialized: now 1790613465491539 us; error 55 us; skew 500 ppm
I20260928 16:37:45.491954 19764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:45 dist-test-slave-9fn6 krb5kdc[19521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613465, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:45.495826 19764 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:37:45.497727 19764 webserver.cc:460] Webserver started at http://127.10.247.194:38599/ using document root <none> and password file <none>
I20260928 16:37:45.498596 19764 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:45.498709 19764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:45.499034 19764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:45.502720 19764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "f4cc6028ae15480aa3cef9f229008927"
format_stamp: "Formatted at 2026-09-28 16:37:45 on dist-test-slave-9fn6"
server_key: "d491d1ca1959f75676ca3254f5e48ec1"
server_key_iv: "a259566908f43b5b4f167730922b9096"
server_key_version: "encryptionkey@0"
I20260928 16:37:45.503685 19764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "f4cc6028ae15480aa3cef9f229008927"
format_stamp: "Formatted at 2026-09-28 16:37:45 on dist-test-slave-9fn6"
server_key: "d491d1ca1959f75676ca3254f5e48ec1"
server_key_iv: "a259566908f43b5b4f167730922b9096"
server_key_version: "encryptionkey@0"
I20260928 16:37:45.510385 19764 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260928 16:37:45.515076 19779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:45.517167 19764 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:37:45.517385 19764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "f4cc6028ae15480aa3cef9f229008927"
format_stamp: "Formatted at 2026-09-28 16:37:45 on dist-test-slave-9fn6"
server_key: "d491d1ca1959f75676ca3254f5e48ec1"
server_key_iv: "a259566908f43b5b4f167730922b9096"
server_key_version: "encryptionkey@0"
I20260928 16:37:45.517637 19764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:45.535205 19764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:45.540940 19764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:45.541333 19764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:45.542654 19764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:45.544401 19764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:45.544493 19764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:45.544662 19764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:45.544730 19764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260928 16:37:45.580578 19764 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:43425
I20260928 16:37:45.580663 19891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:43425 every 8 connection(s)
I20260928 16:37:45.582325 19764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:45.588230 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 19764
I20260928 16:37:45.588446 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:37:45.589018 11231 external_mini_cluster.cc:1428] Setting key febbfbe03373dd7c5ce0187edfcea4eb
Sep 28 16:37:45 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613465, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:45.606698 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:45.585653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:55331 (local address 127.10.247.254:38351)
0928 16:37:45.586269 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:45.586273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:45.587411 (+ 1138us) server_negotiation.cc:407] Connection header received
0928 16:37:45.589681 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:45.589688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:45.589794 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:45.589908 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:45.591871 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:45.592743 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:45.594566 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:45.595089 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:45.595218 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:45.599534 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:45.599541 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:45.599545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:45.599592 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:45.602715 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:45.603866 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:45.603873 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:45.603876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:45.604073 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:45.604583 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:45.604588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:45.604591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:45.605032 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:45.605191 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:45.606055 (+ 864us) server_negotiation.cc:299] Negotiation successful
0928 16:37:45.606324 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":335,"threads_started":1}
I20260928 16:37:45.607952 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:45.586370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:55331)
0928 16:37:45.587211 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:45.587263 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:45.589325 (+ 2062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:45.590241 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:45.590252 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:45.590914 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:45.591672 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:45.591696 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:45.592909 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:45.592914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:45.594388 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:45.594404 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:45.595372 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:45.595380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:45.595681 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:45.596481 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:45.596518 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:45.599203 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:45.603000 (+ 3797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:45.603010 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:45.603029 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:45.603717 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:45.604226 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:45.604230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:45.604232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:45.604466 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:45.605184 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:45.605192 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:45.605639 (+ 447us) client_negotiation.cc:769] Sending connection context
0928 16:37:45.605935 (+ 296us) client_negotiation.cc:240] Negotiation successful
0928 16:37:45.606464 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":308,"threads_started":1}
I20260928 16:37:45.610050 19893 heartbeater.cc:347] Connected to a master server at 127.10.247.254:38351
I20260928 16:37:45.610428 19893 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:45.611387 19893 heartbeater.cc:511] Master 127.10.247.254:38351 requested a full tablet report, sending...
I20260928 16:37:45.613397 19567 ts_manager.cc:194] Registered new tserver with Master: f4cc6028ae15480aa3cef9f229008927 (127.10.247.194:43425)
I20260928 16:37:45.614459 19567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:55331
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:37:45.653060 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38351
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:37:46.029001 19757 heartbeater.cc:503] Master 127.10.247.254:38351 was elected leader, sending a full tablet report...
W20260928 16:37:46.050719 19899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:46.051297 19899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:46.051482 19899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:46.059690 19899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:46.059850 19899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:46.060024 19899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:46.071789 19899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:38351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:46.073559 19899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:46.075662 19899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:46.092010 19904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:46.091961 19905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:46.091928 19907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:46.098697 19899 server_base.cc:1034] running on GCE node
I20260928 16:37:46.099627 19899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:46.101006 19899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:46.102262 19899 hybrid_clock.cc:648] HybridClock initialized: now 1790613466102200 us; error 57 us; skew 500 ppm
I20260928 16:37:46.102662 19899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:46 dist-test-slave-9fn6 krb5kdc[19521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613466, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:46.106559 19899 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:37:46.108500 19899 webserver.cc:460] Webserver started at http://127.10.247.195:38535/ using document root <none> and password file <none>
I20260928 16:37:46.109447 19899 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:46.109570 19899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:46.109944 19899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:37:46.113852 19899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "3e8da80e3004499d93d182674d315abc"
format_stamp: "Formatted at 2026-09-28 16:37:46 on dist-test-slave-9fn6"
server_key: "aa831e7fd60990b6e3f7208b477f45be"
server_key_iv: "4cd1544110c8d1ac02ae1b2b5be908f6"
server_key_version: "encryptionkey@0"
I20260928 16:37:46.114965 19899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "3e8da80e3004499d93d182674d315abc"
format_stamp: "Formatted at 2026-09-28 16:37:46 on dist-test-slave-9fn6"
server_key: "aa831e7fd60990b6e3f7208b477f45be"
server_key_iv: "4cd1544110c8d1ac02ae1b2b5be908f6"
server_key_version: "encryptionkey@0"
I20260928 16:37:46.121886 19899 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:37:46.126459 19914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:46.128261 19899 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:37:46.128463 19899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "3e8da80e3004499d93d182674d315abc"
format_stamp: "Formatted at 2026-09-28 16:37:46 on dist-test-slave-9fn6"
server_key: "aa831e7fd60990b6e3f7208b477f45be"
server_key_iv: "4cd1544110c8d1ac02ae1b2b5be908f6"
server_key_version: "encryptionkey@0"
I20260928 16:37:46.128844 19899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:46.152966 19899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:46.158485 19899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:46.158859 19899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:46.160177 19899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:46.161947 19899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:46.162055 19899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:46.162192 19899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:46.162248 19899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:46.201051 19899 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:40027
I20260928 16:37:46.201138 20026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:40027 every 8 connection(s)
I20260928 16:37:46.203037 19899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:46.203424 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 19899
I20260928 16:37:46.203639 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:37:46.204312 11231 external_mini_cluster.cc:1428] Setting key 80a93455fc23ba9cc9dd0aa16d556f94
Sep 28 16:37:46 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613466, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:37:46.227913 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:46.206218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:50441 (local address 127.10.247.254:38351)
0928 16:37:46.206769 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:46.206776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:46.208059 (+ 1283us) server_negotiation.cc:407] Connection header received
0928 16:37:46.210361 (+ 2302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:46.210367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:46.210461 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:46.210565 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:46.212554 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.213440 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.215197 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.215651 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.215770 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:46.219995 (+ 4225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:46.220001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:46.220005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:46.220049 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:46.223686 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:46.224740 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:46.224747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:46.224751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:46.224918 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:46.225517 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:46.225523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:46.225527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:46.225898 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:46.226086 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:46.227143 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0928 16:37:46.227469 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":303,"threads_started":1}
I20260928 16:37:46.229267 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:46.207040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:50441)
0928 16:37:46.207887 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:46.207940 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:46.210059 (+ 2119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:46.210891 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:46.210901 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:46.211584 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:46.212359 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:46.212385 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.213611 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.213616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:46.215043 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:46.215058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.215904 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.215912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:46.216177 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:46.217012 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:46.217041 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:46.219694 (+ 2653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:37:46.223900 (+ 4206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:46.223911 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:46.223934 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:46.224574 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:46.225055 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:37:46.225066 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:37:46.225070 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:37:46.225349 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:37:46.226060 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:37:46.226070 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:37:46.226654 (+ 584us) client_negotiation.cc:769] Sending connection context
0928 16:37:46.226990 (+ 336us) client_negotiation.cc:240] Negotiation successful
0928 16:37:46.227751 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":356,"threads_started":1}
I20260928 16:37:46.231211 20027 heartbeater.cc:347] Connected to a master server at 127.10.247.254:38351
I20260928 16:37:46.231590 20027 heartbeater.cc:464] Registering TS with master...
I20260928 16:37:46.232527 20027 heartbeater.cc:511] Master 127.10.247.254:38351 requested a full tablet report, sending...
I20260928 16:37:46.234627 19567 ts_manager.cc:194] Registered new tserver with Master: 3e8da80e3004499d93d182674d315abc (127.10.247.195:40027)
I20260928 16:37:46.235858 19567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:50441
I20260928 16:37:46.235909 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab.
Entry for principal oryx/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab.
WARNING: no policy specified for oryx/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab.
Entry for principal oryx/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab.
WARNING: no policy specified for oryx/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab.
Entry for principal oryx/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab.
I20260928 16:37:46.408427 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 19629
Sep 28 16:37:46 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:37:46.428315 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:46.409881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37002 (local address 127.10.247.193:41355)
0928 16:37:46.410683 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:46.410692 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:46.410727 (+ 35us) server_negotiation.cc:407] Connection header received
0928 16:37:46.410809 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:46.410817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:46.411150 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:46.411339 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:46.413363 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.414560 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.416016 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.416750 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.416855 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:46.421786 (+ 4931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:46.421805 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:46.421825 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:46.421873 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:46.425175 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:46.426046 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:46.426057 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:46.426061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:46.426224 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:46.426652 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:46.426655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:46.426657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:46.427107 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:46.427397 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:46.427661 (+ 264us) server_negotiation.cc:299] Negotiation successful
0928 16:37:46.427910 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":414,"threads_started":1}
I20260928 16:37:46.429940 19752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:46.519953 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 19629
I20260928 16:37:46.537762 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 19764
Sep 28 16:37:46 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:37:46.556123 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:46.538814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.10.247.194:43425)
0928 16:37:46.539484 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:46.539493 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:46.539531 (+ 38us) server_negotiation.cc:407] Connection header received
0928 16:37:46.539608 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:46.539616 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:46.539895 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:46.540058 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:46.541895 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.543076 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.544498 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.545363 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.545489 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:46.550180 (+ 4691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:46.550198 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:46.550219 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:46.550267 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:46.553375 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:46.554110 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:46.554117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:46.554119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:46.554255 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:46.554555 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:46.554557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:46.554559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:46.554957 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:46.555229 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:46.555531 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 16:37:46.555777 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":328,"threads_started":1}
I20260928 16:37:46.557804 19887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:46.647887 19893 heartbeater.cc:503] Master 127.10.247.254:38351 was elected leader, sending a full tablet report...
I20260928 16:37:46.649261 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 19764
W20260928 16:37:46.665728 19554 connection.cc:535] server connection from 127.10.247.194:55331 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 16:37:46.666085 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 19899
Sep 28 16:37:46 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:37:46.685559 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:46.667205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41594 (local address 127.10.247.195:40027)
0928 16:37:46.667861 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:46.667869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:46.667905 (+ 36us) server_negotiation.cc:407] Connection header received
0928 16:37:46.668044 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:46.668053 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:46.668337 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:46.668490 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:46.670117 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.671281 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.672760 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:46.673471 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:46.673582 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:46.678832 (+ 5250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:46.678854 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:46.678879 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:46.678945 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:46.682147 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:46.682953 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:46.682962 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:46.682964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:46.683151 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:46.683577 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:46.683582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:46.683586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:46.684109 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:46.684467 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:46.684824 (+ 357us) server_negotiation.cc:299] Negotiation successful
0928 16:37:46.685136 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":311,"threads_started":1}
I20260928 16:37:46.687228 20022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:46.775552 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 19899
I20260928 16:37:46.791538 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 19535
I20260928 16:37:46.793071 19597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:37:46.911592 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 19535
I20260928 16:37:46.932749 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38351
--webserver_interface=127.10.247.254
--webserver_port=40785
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:37:47.325063 20045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:47.325492 20045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:47.325618 20045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:47.333976 20045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:37:47.334141 20045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:47.334192 20045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:47.334254 20045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:37:47.334300 20045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:37:47.345844 20045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:38351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:38351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=40785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:47.347668 20045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:47.349469 20045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:47.363948 20050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:47.363847 20051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:47.363924 20053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:47.364768 20045 server_base.cc:1034] running on GCE node
I20260928 16:37:47.365751 20045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:47.367574 20045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:47.368796 20045 hybrid_clock.cc:648] HybridClock initialized: now 1790613467368726 us; error 54 us; skew 500 ppm
I20260928 16:37:47.369211 20045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:47 dist-test-slave-9fn6 krb5kdc[19521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613467, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:47.372969 20045 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:37:47.374841 20045 webserver.cc:460] Webserver started at http://127.10.247.254:40785/ using document root <none> and password file <none>
I20260928 16:37:47.375823 20045 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:47.375964 20045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:47.383159 20045 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:37:47.386750 20060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:47.388429 20045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:37:47.388652 20045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "42294aed648147e280ea020cb8e4a3ab"
format_stamp: "Formatted at 2026-09-28 16:37:44 on dist-test-slave-9fn6"
server_key: "f6c3b889149a784e35ea2f79ef97d38b"
server_key_iv: "5f6e1413cde7e9a3cb9a936343b2f8df"
server_key_version: "encryptionkey@0"
I20260928 16:37:47.389811 20045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:47.413033 20045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:47.418246 20045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:47.418581 20045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:47.439427 20045 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:38351
I20260928 16:37:47.439452 20112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:38351 every 8 connection(s)
I20260928 16:37:47.441381 20045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:37:47.442130 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 20045
I20260928 16:37:47.443300 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:41355
--local_ip_for_outbound_sockets=127.10.247.193
--tserver_master_addrs=127.10.247.254:38351
--webserver_port=35249
--webserver_interface=127.10.247.193
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.247.193
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:37:47.452143 20113 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:37:47.456897 20113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: Bootstrap starting.
I20260928 16:37:47.480134 20113 log.cc:824] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: Log is configured to *not* fsync() on all Append() calls
I20260928 16:37:47.491803 20113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:37:47.492470 20113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: Bootstrap complete.
I20260928 16:37:47.503301 20113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } }
I20260928 16:37:47.504451 20113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42294aed648147e280ea020cb8e4a3ab, State: Initialized, Role: FOLLOWER
I20260928 16:37:47.505402 20113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } }
I20260928 16:37:47.505728 20113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:37:47.505860 20113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:37:47.506050 20113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 1 FOLLOWER]: Advancing to term 2
I20260928 16:37:47.511152 20113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } }
I20260928 16:37:47.511853 20113 leader_election.cc:302] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42294aed648147e280ea020cb8e4a3ab; no voters:
I20260928 16:37:47.512645 20113 leader_election.cc:288] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 16:37:47.512878 20117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 2 FOLLOWER]: Leader election won for term 2
I20260928 16:37:47.514066 20117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [term 2 LEADER]: Becoming Leader. State: Replica: 42294aed648147e280ea020cb8e4a3ab, State: Running, Role: LEADER
I20260928 16:37:47.514806 20117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } }
I20260928 16:37:47.515401 20113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:37:47.518059 20119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42294aed648147e280ea020cb8e4a3ab. Latest consensus state: current_term: 2 leader_uuid: "42294aed648147e280ea020cb8e4a3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } } }
I20260928 16:37:47.518067 20118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "42294aed648147e280ea020cb8e4a3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42294aed648147e280ea020cb8e4a3ab" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 38351 } } }
I20260928 16:37:47.518538 20119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:47.518544 20118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab [sys.catalog]: This master's current role is: LEADER
I20260928 16:37:47.526865 20124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:37:47.534229 20124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:37:47.536087 20124 catalog_manager.cc:1259] Loaded cluster ID: 30faa1a0fed14f8cb04907d9604780c0
I20260928 16:37:47.536239 20124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:37:47.541129 20124 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:37:47.541332 20134 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 16:37:47.542655 20124 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42294aed648147e280ea020cb8e4a3ab: Loaded TSK: 0
I20260928 16:37:47.544021 20124 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 16:37:47.881182 20115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:47.881628 20115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:47.881786 20115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:47.890547 20115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:47.890726 20115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:47.890900 20115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:37:47.902746 20115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.193.keytab
--principal=oryx/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:41355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=35249
--tserver_master_addrs=127.10.247.254:38351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:47.904706 20115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:47.906435 20115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:47.922518 20140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:47.922638 20142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:47.923338 20139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:47.926573 20115 server_base.cc:1034] running on GCE node
I20260928 16:37:47.927518 20115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:47.929118 20115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:47.930408 20115 hybrid_clock.cc:648] HybridClock initialized: now 1790613467930322 us; error 59 us; skew 500 ppm
I20260928 16:37:47.930869 20115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:47 dist-test-slave-9fn6 krb5kdc[19521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613467, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:47.935721 20115 init.cc:377] Logged in from keytab as oryx/127.10.247.193@KRBTEST.COM (short username oryx)
I20260928 16:37:47.938864 20115 webserver.cc:460] Webserver started at http://127.10.247.193:35249/ using document root <none> and password file <none>
I20260928 16:37:47.939793 20115 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:47.939929 20115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:47.946673 20115 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:37:47.950045 20149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:47.951725 20115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:37:47.951918 20115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "eda316f456f346f19be27cf7a4bf7e87"
format_stamp: "Formatted at 2026-09-28 16:37:44 on dist-test-slave-9fn6"
server_key: "2658381825adb165bc12f458a2b7fef4"
server_key_iv: "567a43882a1b5aed209bd1829562dd09"
server_key_version: "encryptionkey@0"
I20260928 16:37:47.953212 20115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:47.982165 20115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:47.987735 20115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:47.988078 20115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:47.989388 20115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:47.991155 20115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:47.991261 20115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:47.991385 20115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:47.991459 20115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:48.028167 20115 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:41355
I20260928 16:37:48.028236 20262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:41355 every 8 connection(s)
I20260928 16:37:48.030092 20115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:37:48.039870 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 20115
I20260928 16:37:48.040815 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:43425
--local_ip_for_outbound_sockets=127.10.247.194
--tserver_master_addrs=127.10.247.254:38351
--webserver_port=38599
--webserver_interface=127.10.247.194
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.247.194
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 16:37:48.052224 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:48.033676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:40701 (local address 127.10.247.254:38351)
0928 16:37:48.034492 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:48.034507 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:48.034902 (+ 395us) server_negotiation.cc:407] Connection header received
0928 16:37:48.037473 (+ 2571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:48.037488 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:48.037882 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:48.038245 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:48.039942 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.041953 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.043722 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.044754 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.044954 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.050876 (+ 5922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:40701: BlockingRecv error: recv got EOF from 127.10.247.193:40701 (error 108)
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":330,"threads_started":1}
I20260928 16:37:48.052239 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:48.033768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:40701)
0928 16:37:48.034691 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:48.034742 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:48.036858 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:48.038401 (+ 1543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:48.038409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:48.039038 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:48.039687 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.039709 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.042158 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.042162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.043559 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.043574 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.045013 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.045019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.045309 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.046147 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:48.046177 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:48.050431 (+ 4254us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":408,"threads_started":1}
W20260928 16:37:48.053360 20263 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:38351 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.247.254:38351: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:48.063133 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:48.054192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:45369)
0928 16:37:48.054411 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:48.054434 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:48.054586 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:48.054943 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:48.054948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:48.055377 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:48.055588 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.055600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.056788 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.056792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.057923 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.057941 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.058688 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.058694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.058885 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.059557 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:48.059578 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:48.062790 (+ 3212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 16:37:48.063359 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:48.054353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:45369 (local address 127.10.247.254:38351)
0928 16:37:48.054547 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:48.054553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:48.054576 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:37:48.054728 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:48.054732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:48.054805 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:48.054896 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:48.055758 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.056583 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.058072 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.058524 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.058650 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.063106 (+ 4456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:45369: BlockingRecv error: recv got EOF from 127.10.247.193:45369 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:48.072832 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:48.064304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:46377)
0928 16:37:48.064473 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:48.064493 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:48.064601 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:48.065026 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:48.065030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:48.065360 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:48.065538 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.065548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.066619 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.066623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.067635 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.067648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.068374 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.068380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.068563 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.069212 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:48.069232 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:48.072443 (+ 3211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 16:37:48.073207 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:48.064522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:46377 (local address 127.10.247.254:38351)
0928 16:37:48.064749 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:48.064754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:48.064768 (+ 14us) server_negotiation.cc:407] Connection header received
0928 16:37:48.064820 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:48.064824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:48.064887 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:48.064969 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:48.065689 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.066480 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.067775 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.068231 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.068344 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.072779 (+ 4435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:46377: BlockingRecv error: recv got EOF from 127.10.247.193:46377 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260928 16:37:48.073297 20263 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 16:37:48.432116 20267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:48.432513 20267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:48.432650 20267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:48.440784 20267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:48.440922 20267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:48.441107 20267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:37:48.452134 20267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.194.keytab
--principal=oryx/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:43425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=38599
--tserver_master_addrs=127.10.247.254:38351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:48.453898 20267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:48.455494 20267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:48.470310 20272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:48.470254 20273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:48.470273 20275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:48.471155 20267 server_base.cc:1034] running on GCE node
I20260928 16:37:48.471844 20267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:48.472990 20267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:48.474233 20267 hybrid_clock.cc:648] HybridClock initialized: now 1790613468474165 us; error 52 us; skew 500 ppm
I20260928 16:37:48.474596 20267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613468, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:48.478262 20267 init.cc:377] Logged in from keytab as oryx/127.10.247.194@KRBTEST.COM (short username oryx)
I20260928 16:37:48.479945 20267 webserver.cc:460] Webserver started at http://127.10.247.194:38599/ using document root <none> and password file <none>
I20260928 16:37:48.480818 20267 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:48.480966 20267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:48.487340 20267 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:37:48.490455 20282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:48.492141 20267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:37:48.492326 20267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "f4cc6028ae15480aa3cef9f229008927"
format_stamp: "Formatted at 2026-09-28 16:37:45 on dist-test-slave-9fn6"
server_key: "d491d1ca1959f75676ca3254f5e48ec1"
server_key_iv: "a259566908f43b5b4f167730922b9096"
server_key_version: "encryptionkey@0"
I20260928 16:37:48.493477 20267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:48.512683 20267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:48.517666 20267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:48.517953 20267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:48.519117 20267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:48.520714 20267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:48.520839 20267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:48.520962 20267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:48.521034 20267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:48.554710 20267 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:43425
I20260928 16:37:48.554752 20395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:43425 every 8 connection(s)
I20260928 16:37:48.556411 20267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:37:48.562981 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 20267
I20260928 16:37:48.563885 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:40027
--local_ip_for_outbound_sockets=127.10.247.195
--tserver_master_addrs=127.10.247.254:38351
--webserver_port=38535
--webserver_interface=127.10.247.195
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.247.195
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 16:37:48.574858 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:48.559467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:40639 (local address 127.10.247.254:38351)
0928 16:37:48.559692 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:48.559697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:48.561052 (+ 1355us) server_negotiation.cc:407] Connection header received
0928 16:37:48.563116 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:48.563124 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:48.563231 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:48.563379 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:48.565206 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.566302 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.568025 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.568689 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.568834 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.574549 (+ 5715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:40639: BlockingRecv error: recv got EOF from 127.10.247.194:40639 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260928 16:37:48.575812 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:48.560079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:40639)
0928 16:37:48.560873 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:48.560921 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:48.562786 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:48.563590 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:48.563599 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:48.564268 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:48.564990 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.565010 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.566522 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.566526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.567866 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.567880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.568908 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.568913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.569158 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.569917 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:48.569941 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:48.574262 (+ 4321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":278,"threads_started":1}
W20260928 16:37:48.576710 20396 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:38351 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.247.254:38351: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:48.586615 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:48.577396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:48865)
0928 16:37:48.577675 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:48.577693 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:48.577797 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:48.578093 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:48.578098 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:48.578517 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:48.578721 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.578732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.579845 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.579850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.581103 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.581118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.581990 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.581995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.582149 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.582754 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:48.582776 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:48.586283 (+ 3507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156}
W20260928 16:37:48.586764 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:48.577621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:48865 (local address 127.10.247.254:38351)
0928 16:37:48.577797 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:48.577802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:48.577821 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:37:48.577901 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:48.577905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:48.577975 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:48.578054 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:48.578876 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.579709 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.581264 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.581842 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.581967 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.586518 (+ 4551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:48865: BlockingRecv error: recv got EOF from 127.10.247.194:48865 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:48 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:48.596549 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:48.587378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:43317)
0928 16:37:48.587651 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:48.587670 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:48.587792 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:48.588122 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:48.588126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:48.588511 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:48.588751 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.588764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.589794 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.589800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.591281 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:48.591296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.592030 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.592036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:48.592269 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.592903 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:48.592929 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:48.596292 (+ 3363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160}
W20260928 16:37:48.596932 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:48.587510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:43317 (local address 127.10.247.254:38351)
0928 16:37:48.587680 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:48.587684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:48.587723 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:37:48.587898 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:48.587902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:48.587988 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:48.588070 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:48.588885 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.589659 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.591432 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:48.591883 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:48.591980 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:48.596663 (+ 4683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:43317: BlockingRecv error: recv got EOF from 127.10.247.194:43317 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260928 16:37:48.596920 20396 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 16:37:48.949303 20399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:37:48.949679 20399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:37:48.949829 20399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:37:48.957772 20399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:37:48.957918 20399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:37:48.958089 20399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:37:48.968922 20399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/krb5kdc/oryx_127.10.247.195.keytab
--principal=oryx/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:40027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=38535
--tserver_master_addrs=127.10.247.254:38351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:37:48.970697 20399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:37:48.972366 20399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:37:48.990096 20406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:48.990523 20405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:37:48.990163 20408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:37:48.994185 20399 server_base.cc:1034] running on GCE node
I20260928 16:37:48.995235 20399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:37:48.996892 20399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:37:48.998178 20399 hybrid_clock.cc:648] HybridClock initialized: now 1790613468998097 us; error 67 us; skew 500 ppm
I20260928 16:37:48.998729 20399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613469, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:37:49.003598 20399 init.cc:377] Logged in from keytab as oryx/127.10.247.195@KRBTEST.COM (short username oryx)
I20260928 16:37:49.005867 20399 webserver.cc:460] Webserver started at http://127.10.247.195:38535/ using document root <none> and password file <none>
I20260928 16:37:49.007050 20399 fs_manager.cc:359] Metadata directory not provided
I20260928 16:37:49.007215 20399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:37:49.015151 20399 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:37:49.019639 20415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:49.021941 20399 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260928 16:37:49.022161 20399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "3e8da80e3004499d93d182674d315abc"
format_stamp: "Formatted at 2026-09-28 16:37:46 on dist-test-slave-9fn6"
server_key: "aa831e7fd60990b6e3f7208b477f45be"
server_key_iv: "4cd1544110c8d1ac02ae1b2b5be908f6"
server_key_version: "encryptionkey@0"
I20260928 16:37:49.023720 20399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:37:49.055336 20399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:37:49.060420 20399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:37:49.060757 20399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:37:49.061949 20399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:37:49.063494 20399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:37:49.063607 20399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:37:49.063719 20399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:37:49.063787 20399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:49.086315 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:49.074138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:42179)
0928 16:37:49.075106 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:49.075133 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:49.075315 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:49.075840 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:49.075848 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:49.076451 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:49.076796 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.076813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.078242 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.078250 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.079878 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.079898 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.080880 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.080887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.081073 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.081853 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:49.081881 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:49.085867 (+ 3986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":656,"threads_started":1}
W20260928 16:37:49.086468 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:49.074324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:42179 (local address 127.10.247.254:38351)
0928 16:37:49.074628 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:49.074637 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:49.075330 (+ 693us) server_negotiation.cc:407] Connection header received
0928 16:37:49.075504 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:49.075510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:49.075609 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:49.075751 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:49.077004 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.078053 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.080045 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.080674 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.080801 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.086146 (+ 5345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:42179: BlockingRecv error: recv got EOF from 127.10.247.193:42179 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
I20260928 16:37:49.105046 20399 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:40027
I20260928 16:37:49.105120 20529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:40027 every 8 connection(s)
I20260928 16:37:49.106863 20399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:37:49.116695 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 20399
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 16:37:49.125383 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:49.110048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:58043 (local address 127.10.247.254:38351)
0928 16:37:49.110251 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:49.110256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:49.111384 (+ 1128us) server_negotiation.cc:407] Connection header received
0928 16:37:49.113496 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:49.113503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:49.113603 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:49.113703 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:49.115479 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.116302 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.118620 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.119333 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.119459 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.125075 (+ 5616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:58043: BlockingRecv error: recv got EOF from 127.10.247.195:58043 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 16:37:49.126415 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:49.110424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:58043)
0928 16:37:49.111213 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:49.111254 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:49.113189 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:49.113981 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:49.113991 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:49.114598 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:49.115290 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.115309 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.116467 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.116471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.118381 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.118403 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.119586 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.119594 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.119881 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.120778 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:49.120807 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:49.124869 (+ 4062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":295,"threads_started":1}
W20260928 16:37:49.127223 20530 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:38351 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.247.254:38351: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:49.135030 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:49.117883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36760 (local address 127.10.247.254:38351)
0928 16:37:49.118596 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:49.118602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:49.119335 (+ 733us) server_negotiation.cc:407] Connection header received
0928 16:37:49.119531 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:49.119537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:49.119627 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:49.119726 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:49.121633 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.122556 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.123931 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.124387 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.124498 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.127751 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:37:49.127766 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:37:49.127793 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:37:49.127876 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:37:49.131452 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:49.132175 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:49.132183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:49.132191 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:49.132416 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:37:49.132862 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:37:49.132867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:37:49.132869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:37:49.133423 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:37:49.133762 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0928 16:37:49.134276 (+ 514us) server_negotiation.cc:299] Negotiation successful
0928 16:37:49.134702 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":329,"threads_started":1}
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:49.136253 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:49.127949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:60547)
0928 16:37:49.128157 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:49.128180 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:49.128302 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:49.128785 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:49.128789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:49.129163 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:49.129345 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.129354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.130534 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.130538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.131501 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.131515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.132177 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.132183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.132345 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.132874 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:49.132892 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:49.135972 (+ 3080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260928 16:37:49.136545 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:49.128048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:60547 (local address 127.10.247.254:38351)
0928 16:37:49.128365 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:49.128371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:49.128405 (+ 34us) server_negotiation.cc:407] Connection header received
0928 16:37:49.128521 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:49.128525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:49.128593 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:49.128746 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:49.129487 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.130395 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.131636 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.132041 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.132146 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.136155 (+ 4009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:60547: BlockingRecv error: recv got EOF from 127.10.247.195:60547 (error 108)
Metrics: {"server-negotiator.queue_time_us":186}
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:49.145187 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:49.137080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:52189)
0928 16:37:49.137256 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:49.137275 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:49.137379 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:49.137702 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:49.137707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:49.138031 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:49.138205 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.138218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.139328 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.139333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.140432 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.140446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.141187 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.141190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.141323 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.141851 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:49.141867 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:49.144917 (+ 3050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 16:37:49.145375 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:49.137161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:52189 (local address 127.10.247.254:38351)
0928 16:37:49.137383 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:49.137388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:49.137405 (+ 17us) server_negotiation.cc:407] Connection header received
0928 16:37:49.137506 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:49.137511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:49.137580 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:49.137660 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:49.138347 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.139187 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.140553 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.141064 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.141157 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.145150 (+ 3993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:52189: BlockingRecv error: recv got EOF from 127.10.247.195:52189 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
W20260928 16:37:49.145598 20530 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:49 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:49.607324 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:49.597606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:55675)
0928 16:37:49.598294 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:49.598312 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:49.598485 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:49.598841 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:49.598847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:49.599446 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:49.599689 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.599701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.600863 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.600870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.601931 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:49.601948 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.602770 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.602777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:49.602969 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.603687 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:49.603708 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:49.606950 (+ 3242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":369,"threads_started":1}
W20260928 16:37:49.607523 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:49.597705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:55675 (local address 127.10.247.254:38351)
0928 16:37:49.597949 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:49.597957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:49.598404 (+ 447us) server_negotiation.cc:407] Connection header received
0928 16:37:49.598617 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:49.598623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:49.598704 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:49.598789 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:49.599845 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.600707 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.602085 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:49.602568 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:49.602687 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:49.607243 (+ 4556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:55675: BlockingRecv error: recv got EOF from 127.10.247.194:55675 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 28 16:37:50 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:50 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:50.098800 20536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:50.087772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:45351)
0928 16:37:50.088400 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:50.088418 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:50.088596 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:50.089032 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:50.089037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:50.089561 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:50.089777 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:50.089789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.090991 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.090996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:50.092467 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:50.092485 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.093765 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.093772 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:50.093973 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:50.094784 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:50.094811 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:50.098486 (+ 3675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":350,"threads_started":1}
W20260928 16:37:50.099124 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:50.087896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:45351 (local address 127.10.247.254:38351)
0928 16:37:50.088170 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:50.088178 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:50.088541 (+ 363us) server_negotiation.cc:407] Connection header received
0928 16:37:50.088778 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:50.088783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:50.088862 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:50.088956 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:50.089953 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.090798 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.092647 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.093364 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.093500 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:50.098830 (+ 5330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:45351: BlockingRecv error: recv got EOF from 127.10.247.193:45351 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 28 16:37:50 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:50 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:50.155676 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:50.146329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:54329)
0928 16:37:50.146917 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:50.146934 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:50.147110 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:50.147481 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:50.147486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:50.148035 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:50.148264 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:50.148275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.149437 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.149442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:50.150469 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:50.150482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.151281 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.151287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:50.151451 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:50.152123 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:50.152141 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:50.155310 (+ 3169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":313,"threads_started":1}
W20260928 16:37:50.155896 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:50.146418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:54329 (local address 127.10.247.254:38351)
0928 16:37:50.146675 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:50.146682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:50.147071 (+ 389us) server_negotiation.cc:407] Connection header received
0928 16:37:50.147247 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:50.147251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:50.147335 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:50.147420 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:50.148423 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.149274 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.150614 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.151094 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.151212 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:50.155584 (+ 4372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:54329: BlockingRecv error: recv got EOF from 127.10.247.195:54329 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 28 16:37:50 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:50 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:50.618192 20538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:50.608497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:35605)
0928 16:37:50.609163 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:50.609181 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:50.609321 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:50.609675 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:50.609679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:50.610175 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:50.610386 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:50.610398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.611488 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.611491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:50.612470 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:50.612485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.613371 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.613376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:50.613524 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:50.614270 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:50.614292 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:50.617791 (+ 3499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":341,"threads_started":1}
W20260928 16:37:50.618382 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:50.608634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:35605 (local address 127.10.247.254:38351)
0928 16:37:50.608856 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:50.608862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:50.609325 (+ 463us) server_negotiation.cc:407] Connection header received
0928 16:37:50.609442 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:50.609448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:50.609541 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:50.609645 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:50.610551 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.611364 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.612659 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:50.613160 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:50.613279 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:50.618129 (+ 4850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:35605: BlockingRecv error: recv got EOF from 127.10.247.194:35605 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 28 16:37:51 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:51 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:51.110255 20539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:51.099928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:47695)
0928 16:37:51.100635 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:51.100656 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:51.100805 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:51.101175 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:51.101179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:51.101593 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:51.101799 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:51.101810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.102962 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.102967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:51.103951 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:51.103965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.104882 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.104888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:51.105156 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:51.105976 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:51.106001 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:51.109915 (+ 3914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":416,"threads_started":1}
W20260928 16:37:51.110378 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:51.100208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:47695 (local address 127.10.247.254:38351)
0928 16:37:51.100404 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:51.100410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:51.100726 (+ 316us) server_negotiation.cc:407] Connection header received
0928 16:37:51.100924 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:51.100929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:51.101033 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:51.101115 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:51.101944 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.102778 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.104129 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.104713 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.104840 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:51.110140 (+ 5300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:47695: BlockingRecv error: recv got EOF from 127.10.247.193:47695 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 28 16:37:51 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:51 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:51.166726 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:51.157071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:43865)
0928 16:37:51.157746 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:51.157765 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:51.157929 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:51.158430 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:51.158434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:51.159077 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:51.159301 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:51.159316 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.160722 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.160727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:51.161708 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:51.161721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.162387 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.162393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:51.162522 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:51.163193 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:51.163212 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:51.166432 (+ 3220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":361,"threads_started":1}
W20260928 16:37:51.167048 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:51.157216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:43865 (local address 127.10.247.254:38351)
0928 16:37:51.157483 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:51.157488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:51.157878 (+ 390us) server_negotiation.cc:407] Connection header received
0928 16:37:51.158099 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:51.158104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:51.158194 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:51.158348 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:51.159477 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.160527 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.161830 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.162243 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.162341 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:51.166732 (+ 4391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:43865: BlockingRecv error: recv got EOF from 127.10.247.195:43865 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 28 16:37:51 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:51 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:51.629892 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:51.619518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:48183)
0928 16:37:51.620150 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:51.620169 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:51.620343 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:51.620782 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:51.620787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:51.621254 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:51.621491 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:51.621504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.622714 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.622719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:51.623734 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:51.623749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.624582 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.624589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:51.624841 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:51.625599 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:51.625621 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:51.629562 (+ 3941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":333,"threads_started":1}
W20260928 16:37:51.630050 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:51.619670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:48183 (local address 127.10.247.254:38351)
0928 16:37:51.619986 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:51.619994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:51.620269 (+ 275us) server_negotiation.cc:407] Connection header received
0928 16:37:51.620497 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:51.620503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:51.620581 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:51.620727 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:51.621662 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.622555 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.623890 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:51.624409 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:51.624573 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:51.629815 (+ 5242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:48183: BlockingRecv error: recv got EOF from 127.10.247.194:48183 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 28 16:37:52 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:52 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:52.120414 20542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:52.111405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:55089)
0928 16:37:52.111980 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:52.111997 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:52.112179 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:52.112529 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:52.112534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:52.113028 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:52.113222 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:52.113233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.114290 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.114296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:52.115343 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:52.115360 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.116111 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.116117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:52.116293 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:52.116987 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:52.117007 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:52.120122 (+ 3115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":315,"threads_started":1}
W20260928 16:37:52.120656 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:52.111487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:55089 (local address 127.10.247.254:38351)
0928 16:37:52.111699 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:52.111706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:52.112113 (+ 407us) server_negotiation.cc:407] Connection header received
0928 16:37:52.112316 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:52.112320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:52.112392 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:52.112477 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:52.113364 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.114161 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.115471 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.115962 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.116075 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:52.120365 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:55089: BlockingRecv error: recv got EOF from 127.10.247.193:55089 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 28 16:37:52 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:52 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:52.177502 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:52.167867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:54003)
0928 16:37:52.168425 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:52.168445 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:52.168658 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:52.169042 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:52.169048 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:52.169549 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:52.169784 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:52.169800 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.171143 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.171148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:52.172162 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:52.172179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.173042 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.173047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:52.173239 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:52.173914 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:52.173934 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:52.177194 (+ 3260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":515,"threads_started":1}
W20260928 16:37:52.177731 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:52.167957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:54003 (local address 127.10.247.254:38351)
0928 16:37:52.168233 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:52.168241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:52.168547 (+ 306us) server_negotiation.cc:407] Connection header received
0928 16:37:52.168801 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:52.168806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:52.168884 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:52.168983 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:52.169977 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.171001 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.172315 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.172870 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.172974 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:52.177479 (+ 4505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:54003: BlockingRecv error: recv got EOF from 127.10.247.195:54003 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Sep 28 16:37:52 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:52 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:52.645972 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:52.630996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:41397)
0928 16:37:52.631689 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:52.631707 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:52.631850 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:52.632199 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:52.632202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:52.632697 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:52.632889 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:52.632900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.634155 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.634160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:52.635402 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:52.635421 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.636429 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.636436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:52.636664 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:52.642202 (+ 5538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:52.642227 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:52.645656 (+ 3429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":351,"threads_started":1}
W20260928 16:37:52.646219 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:52.631121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:41397 (local address 127.10.247.254:38351)
0928 16:37:52.631342 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:52.631348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:52.631801 (+ 453us) server_negotiation.cc:407] Connection header received
0928 16:37:52.631997 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:52.632001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:52.632078 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:52.632168 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:52.633062 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.633939 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.635568 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:52.636223 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:52.636349 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:52.645933 (+ 9584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:41397: BlockingRecv error: recv got EOF from 127.10.247.194:41397 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 28 16:37:53 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:53 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:53.131592 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:53.121732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:50919)
0928 16:37:53.122411 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:53.122429 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:53.122594 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:53.122971 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:53.122976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:53.123487 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:53.123692 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:53.123703 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.124943 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.124948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:53.126087 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:53.126106 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.126982 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.126990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:53.127218 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:53.127972 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:53.128000 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:53.131286 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":353,"threads_started":1}
W20260928 16:37:53.131793 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:53.121896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:50919 (local address 127.10.247.254:38351)
0928 16:37:53.122162 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:53.122170 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:53.122530 (+ 360us) server_negotiation.cc:407] Connection header received
0928 16:37:53.122733 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:53.122739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:53.122814 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:53.122908 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:53.123853 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.124794 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.126232 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.126771 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.126909 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:53.131535 (+ 4626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:50919: BlockingRecv error: recv got EOF from 127.10.247.193:50919 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 28 16:37:53 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:53 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:53.188369 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:53.178688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:36871)
0928 16:37:53.179440 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:53.179458 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:53.179607 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:53.180094 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:53.180099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:53.180575 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:53.180802 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:53.180814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.182044 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.182049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:53.183123 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:53.183140 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.183970 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.183975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:53.184131 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:53.184867 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:53.184888 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:53.188090 (+ 3202us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":469,"threads_started":1}
W20260928 16:37:53.188591 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:53.178805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:36871 (local address 127.10.247.254:38351)
0928 16:37:53.179142 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:53.179149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:53.179534 (+ 385us) server_negotiation.cc:407] Connection header received
0928 16:37:53.179785 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:53.179790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:53.179871 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:53.179956 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:53.181004 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.181874 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.183276 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.183803 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.183908 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:53.188347 (+ 4439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:36871: BlockingRecv error: recv got EOF from 127.10.247.195:36871 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 28 16:37:53 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:53 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:53.656663 20547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:53.647218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:60811)
0928 16:37:53.647843 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:53.647862 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:53.647996 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:53.648373 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:53.648378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:53.648964 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:53.649173 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:53.649184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.650259 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.650262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:53.651207 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:53.651220 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.651980 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.651984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:53.652117 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:53.652803 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:53.652820 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:53.656244 (+ 3424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":306,"threads_started":1}
W20260928 16:37:53.656827 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:53.647368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:60811 (local address 127.10.247.254:38351)
0928 16:37:53.647550 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:53.647556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:53.647945 (+ 389us) server_negotiation.cc:407] Connection header received
0928 16:37:53.648121 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:53.648125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:53.648198 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:53.648283 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:53.649314 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.650138 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.651368 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:53.651848 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:53.651993 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:53.656487 (+ 4494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:60811: BlockingRecv error: recv got EOF from 127.10.247.194:60811 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 28 16:37:54 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:54 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:54.142107 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:54.132718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:51253)
0928 16:37:54.133336 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:54.133354 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:54.133534 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:54.133886 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:54.133890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:54.134335 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:54.134539 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:54.134550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.135672 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.135677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:54.136705 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:54.136722 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.137622 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.137627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:54.137785 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:54.138523 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:54.138551 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:54.141793 (+ 3242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":334,"threads_started":1}
W20260928 16:37:54.142315 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:54.132796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:51253 (local address 127.10.247.254:38351)
0928 16:37:54.133054 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:54.133061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:54.133449 (+ 388us) server_negotiation.cc:407] Connection header received
0928 16:37:54.133669 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:54.133673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:54.133749 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:54.133838 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:54.134700 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.135521 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.136853 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.137456 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.137606 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:54.142065 (+ 4459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:51253: BlockingRecv error: recv got EOF from 127.10.247.193:51253 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 28 16:37:54 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:54 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:54.199450 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:54.189656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:38589)
0928 16:37:54.190441 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:54.190458 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:54.190588 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:54.190885 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:54.190888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:54.191404 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:54.191608 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:54.191620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.192828 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.192833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:54.193975 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:54.193993 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.194807 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.194812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:54.195030 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:54.195740 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:54.195764 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:54.199151 (+ 3387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":473,"threads_started":1}
W20260928 16:37:54.199738 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:54.189759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:38589 (local address 127.10.247.254:38351)
0928 16:37:54.190017 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:54.190024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:54.190547 (+ 523us) server_negotiation.cc:407] Connection header received
0928 16:37:54.190692 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:54.190696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:54.190768 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:54.190855 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:54.191772 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.192647 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.194147 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.194635 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.194744 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:54.199449 (+ 4705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:38589: BlockingRecv error: recv got EOF from 127.10.247.195:38589 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 28 16:37:54 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:54 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:54.667469 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:54.657888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:37937)
0928 16:37:54.658523 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:54.658540 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:54.658717 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:54.659113 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:54.659118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:54.659578 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:54.659767 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:54.659778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.660923 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.660928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:54.661877 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:54.661889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.662657 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.662663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:54.662844 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:54.663523 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:54.663544 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:54.667198 (+ 3654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":361,"threads_started":1}
W20260928 16:37:54.667742 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:54.657996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:37937 (local address 127.10.247.254:38351)
0928 16:37:54.658259 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:54.658266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:54.658646 (+ 380us) server_negotiation.cc:407] Connection header received
0928 16:37:54.658845 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:54.658850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:54.658961 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:54.659052 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:54.659911 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.660791 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.662019 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:54.662501 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:54.662613 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:54.667476 (+ 4863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:37937: BlockingRecv error: recv got EOF from 127.10.247.194:37937 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 28 16:37:55 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:55 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:55.153059 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:55.143221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:34875)
0928 16:37:55.143961 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:55.143982 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:55.144171 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:55.144497 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:55.144501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:55.145070 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:55.145296 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:55.145308 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.146519 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.146523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:55.147506 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:55.147519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.148453 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.148460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:55.148686 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:55.149416 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:55.149437 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:55.152649 (+ 3212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":341,"threads_started":1}
W20260928 16:37:55.153209 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:55.143309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:34875 (local address 127.10.247.254:38351)
0928 16:37:55.143542 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:55.143550 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:55.144081 (+ 531us) server_negotiation.cc:407] Connection header received
0928 16:37:55.144284 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:55.144288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:55.144360 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:55.144445 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:55.145456 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.146373 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.147641 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.148285 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.148403 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:55.152928 (+ 4525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:34875: BlockingRecv error: recv got EOF from 127.10.247.193:34875 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 28 16:37:55 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:55 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:55.209604 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:55.200659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:34839)
0928 16:37:55.201305 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:55.201322 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:55.201474 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:55.201847 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:55.201852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:55.202308 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:55.202503 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:55.202513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.203607 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.203613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:55.204587 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:55.204601 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.205341 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.205348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:55.205529 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:55.206202 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:55.206221 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:55.209340 (+ 3119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":369,"threads_started":1}
W20260928 16:37:55.209815 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:55.200748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:34839 (local address 127.10.247.254:38351)
0928 16:37:55.201000 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:55.201007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:55.201411 (+ 404us) server_negotiation.cc:407] Connection header received
0928 16:37:55.201611 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:55.201617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:55.201696 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:55.201788 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:55.202663 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.203465 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.204751 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.205187 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.205300 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:55.209559 (+ 4259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:34839: BlockingRecv error: recv got EOF from 127.10.247.195:34839 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 28 16:37:55 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:55 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:55.677572 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:55.668466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:39491)
0928 16:37:55.669205 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:55.669223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:55.669375 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:55.669730 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:55.669735 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:55.670234 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:55.670435 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:55.670446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.671597 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.671601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:55.672574 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:55.672587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.673368 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.673375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:55.673546 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:55.674180 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:55.674200 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:55.677200 (+ 3000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":277,"threads_started":1}
W20260928 16:37:55.677754 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:55.668549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:39491 (local address 127.10.247.254:38351)
0928 16:37:55.668859 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:55.668867 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:55.669314 (+ 447us) server_negotiation.cc:407] Connection header received
0928 16:37:55.669496 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:55.669500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:55.669587 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:55.669676 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:55.670593 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.671453 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.672759 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:55.673212 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:55.673325 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:55.677455 (+ 4130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:39491: BlockingRecv error: recv got EOF from 127.10.247.194:39491 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
Sep 28 16:37:56 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:56 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:56.164793 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:56.154204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:35347)
0928 16:37:56.154787 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:56.154806 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:56.154952 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:56.155303 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:56.155306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:56.155757 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:56.155953 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:56.155963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.157320 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.157338 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:56.158802 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:56.158826 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.160025 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.160031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:56.160179 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:56.160959 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:56.160984 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:56.164437 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":324,"threads_started":1}
W20260928 16:37:56.164995 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:56.154286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:35347 (local address 127.10.247.254:38351)
0928 16:37:56.154553 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:56.154561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:56.154894 (+ 333us) server_negotiation.cc:407] Connection header received
0928 16:37:56.155102 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:56.155106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:56.155184 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:56.155271 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:56.156170 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.157097 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.159088 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.159790 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.159923 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:56.164778 (+ 4855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:35347: BlockingRecv error: recv got EOF from 127.10.247.193:35347 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 28 16:37:56 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:56 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 16:37:56.221915 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:56.210823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:36579 (local address 127.10.247.254:38351)
0928 16:37:56.211049 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:56.211058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:56.211451 (+ 393us) server_negotiation.cc:407] Connection header received
0928 16:37:56.211638 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:56.211643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:56.211713 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:56.211798 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:56.212848 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.213968 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.215506 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.216014 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.216131 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:56.221601 (+ 5470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:36579: BlockingRecv error: recv got EOF from 127.10.247.195:36579 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:37:56.222009 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:56.210666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:36579)
0928 16:37:56.211347 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:56.211365 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:56.211516 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:56.211883 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:56.211888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:56.212402 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:56.212689 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:56.212701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.214137 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.214144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:56.215376 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:56.215393 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.216174 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.216179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:56.216350 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:56.217100 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:56.217127 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:56.221325 (+ 4198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":315,"threads_started":1}
Sep 28 16:37:56 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:56 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:56.688861 20556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:56.678965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:57957)
0928 16:37:56.679600 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:56.679618 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:56.679788 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:56.680171 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:56.680174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:56.680780 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:56.681033 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:56.681049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.682435 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.682440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:56.683488 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:56.683503 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.684289 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.684295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:56.684475 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:56.685201 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:56.685222 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:56.688513 (+ 3291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":331,"threads_started":1}
W20260928 16:37:56.689162 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:56.679067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:57957 (local address 127.10.247.254:38351)
0928 16:37:56.679296 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:56.679303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:56.679720 (+ 417us) server_negotiation.cc:407] Connection header received
0928 16:37:56.679933 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:56.679939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:56.680018 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:56.680116 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:56.681208 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.682276 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.683633 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:56.684091 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:56.684199 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:56.688864 (+ 4665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:57957: BlockingRecv error: recv got EOF from 127.10.247.194:57957 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 28 16:37:57 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:57 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:57.175979 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:57.166041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:44813)
0928 16:37:57.166771 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:57.166790 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:57.166945 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:57.167388 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:57.167394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:57.167962 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:57.168206 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:57.168222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.169541 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.169546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:57.170555 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:57.170568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.171420 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.171425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:57.171575 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:57.172260 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:57.172279 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:57.175628 (+ 3349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":422,"threads_started":1}
W20260928 16:37:57.176276 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:57.166199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:44813 (local address 127.10.247.254:38351)
0928 16:37:57.166463 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:57.166471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:57.166890 (+ 419us) server_negotiation.cc:407] Connection header received
0928 16:37:57.167087 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:57.167092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:57.167218 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:57.167331 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:57.168409 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.169357 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.170710 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.171270 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.171381 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:57.175952 (+ 4571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:44813: BlockingRecv error: recv got EOF from 127.10.247.193:44813 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 28 16:37:57 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:57 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:57.232995 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:57.223170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:42503)
0928 16:37:57.223801 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:57.223819 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:57.223960 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:57.224283 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:57.224286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:57.224794 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:57.224988 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:57.224998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.226182 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.226187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:57.227314 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:57.227330 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.228198 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.228202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:57.228330 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:57.229007 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:57.229028 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:57.232698 (+ 3670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":327,"threads_started":1}
W20260928 16:37:57.233232 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:57.223276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:42503 (local address 127.10.247.254:38351)
0928 16:37:57.223545 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:57.223550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:57.223900 (+ 350us) server_negotiation.cc:407] Connection header received
0928 16:37:57.224067 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:57.224071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:57.224159 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:57.224249 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:57.225181 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.226008 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.227452 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.228012 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.228152 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:57.232928 (+ 4776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:42503: BlockingRecv error: recv got EOF from 127.10.247.195:42503 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Sep 28 16:37:57 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:57 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:57.704759 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:57.690280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:50733)
0928 16:37:57.690925 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:57.690944 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:57.691129 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:57.691540 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:57.691546 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:57.692240 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:57.692525 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:57.692542 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.693802 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.693807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:57.694915 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:57.694933 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.695747 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.695752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:57.695942 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:57.700410 (+ 4468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:57.700438 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:57.704372 (+ 3934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":595,"threads_started":1}
W20260928 16:37:57.705497 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:57.690438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:50733 (local address 127.10.247.254:38351)
0928 16:37:57.690701 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:57.690708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:57.691048 (+ 340us) server_negotiation.cc:407] Connection header received
0928 16:37:57.691262 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:57.691268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:57.691364 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:57.691473 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:57.692705 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.693655 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.695064 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:57.695577 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:57.695698 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:57.704789 (+ 9091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:50733: BlockingRecv error: recv got EOF from 127.10.247.194:50733 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Sep 28 16:37:58 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:58 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:58.188086 20560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:58.177431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:55261)
0928 16:37:58.178089 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:58.178120 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:58.178303 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:58.178787 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:58.178793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:58.179457 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:58.179700 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:58.179716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.181159 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.181164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:58.182293 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:58.182311 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.183224 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.183231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:58.183414 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:58.184181 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:58.184202 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:58.187780 (+ 3578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":370,"threads_started":1}
W20260928 16:37:58.188351 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:58.177613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:55261 (local address 127.10.247.254:38351)
0928 16:37:58.177915 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:58.177923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:58.178223 (+ 300us) server_negotiation.cc:407] Connection header received
0928 16:37:58.178458 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:58.178464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:58.178601 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:58.178727 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:58.179919 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.181000 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.182469 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.183002 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.183131 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:58.188083 (+ 4952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:55261: BlockingRecv error: recv got EOF from 127.10.247.193:55261 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Sep 28 16:37:58 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:58 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:58.244246 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:58.234178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:54883)
0928 16:37:58.234856 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:58.234874 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:58.235061 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:58.235505 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:58.235510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:58.236079 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:58.236318 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:58.236331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.237591 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.237596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:58.238659 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:58.238676 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.239568 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.239573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:58.239739 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:58.240509 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:58.240532 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:58.243929 (+ 3397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":359,"threads_started":1}
W20260928 16:37:58.244524 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:58.234364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:54883 (local address 127.10.247.254:38351)
0928 16:37:58.234592 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:58.234598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:58.234967 (+ 369us) server_negotiation.cc:407] Connection header received
0928 16:37:58.235219 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:58.235223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:58.235315 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:58.235406 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:58.236492 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.237418 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.238851 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.239378 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.239507 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:58.244219 (+ 4712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:54883: BlockingRecv error: recv got EOF from 127.10.247.195:54883 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 28 16:37:58 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:58 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:58.716908 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:58.706248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:60145)
0928 16:37:58.706922 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:58.706942 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:58.707090 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:58.707548 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:58.707553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:58.708048 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:58.708237 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:58.708247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.709442 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.709448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:58.710911 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:58.710929 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.711934 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.711940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:58.712078 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:58.712950 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:58.712978 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:58.716444 (+ 3466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":392,"threads_started":1}
W20260928 16:37:58.717083 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:58.706421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:60145 (local address 127.10.247.254:38351)
0928 16:37:58.706633 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:58.706639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:58.707036 (+ 397us) server_negotiation.cc:407] Connection header received
0928 16:37:58.707237 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:58.707242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:58.707318 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:58.707410 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:58.708383 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.709273 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.711149 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:58.711720 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:58.711826 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:58.716785 (+ 4959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:60145: BlockingRecv error: recv got EOF from 127.10.247.194:60145 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 28 16:37:59 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:59 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:59.199749 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:59.189693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:55913)
0928 16:37:59.190403 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:59.190421 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:59.190590 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:59.191020 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:59.191026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:59.191606 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:59.191896 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:59.191911 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.193148 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.193153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:59.194305 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:59.194321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.195203 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.195209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:59.195359 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:59.196051 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:59.196073 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:59.199430 (+ 3357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":394,"threads_started":1}
W20260928 16:37:59.199990 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:59.189949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:55913 (local address 127.10.247.254:38351)
0928 16:37:59.190248 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:59.190255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:59.190535 (+ 280us) server_negotiation.cc:407] Connection header received
0928 16:37:59.190766 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:59.190770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:59.190848 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:59.190943 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:59.192056 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.193019 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.194473 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.195034 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.195160 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:59.199721 (+ 4561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:55913: BlockingRecv error: recv got EOF from 127.10.247.193:55913 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
Sep 28 16:37:59 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:59 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:59.255136 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:59.245416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:36339)
0928 16:37:59.246137 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:59.246160 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:59.246325 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:59.246799 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:59.246803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:59.247250 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:59.247426 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:59.247435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.248575 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.248580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:59.249663 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:59.249679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.250517 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.250521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:59.250653 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:59.251329 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:59.251349 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:59.254785 (+ 3436us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":386,"threads_started":1}
W20260928 16:37:59.255645 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:59.245576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:36339 (local address 127.10.247.254:38351)
0928 16:37:59.245932 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:59.245939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:59.246241 (+ 302us) server_negotiation.cc:407] Connection header received
0928 16:37:59.246525 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:59.246533 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:59.246654 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:59.246756 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:59.247567 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.248392 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.249829 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.250344 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.250453 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:59.255098 (+ 4645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:36339: BlockingRecv error: recv got EOF from 127.10.247.195:36339 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 28 16:37:59 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:37:59 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:37:59.727883 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:37:59.718011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:45541)
0928 16:37:59.718623 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0928 16:37:59.718643 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:37:59.718784 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:37:59.719204 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:37:59.719209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:37:59.719790 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:37:59.720077 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:59.720090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.721466 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.721471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:59.722448 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:37:59.722463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.723331 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.723336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:37:59.723483 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:59.724170 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:37:59.724189 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:37:59.727532 (+ 3343us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":305,"threads_started":1}
W20260928 16:37:59.728080 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:37:59.718080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:45541 (local address 127.10.247.254:38351)
0928 16:37:59.718288 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 16:37:59.718293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:37:59.718717 (+ 424us) server_negotiation.cc:407] Connection header received
0928 16:37:59.718898 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:37:59.718902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:37:59.718976 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:37:59.719063 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:37:59.720311 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.721291 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.722611 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:37:59.723169 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:37:59.723288 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:37:59.727803 (+ 4515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:45541: BlockingRecv error: recv got EOF from 127.10.247.194:45541 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 28 16:38:00 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:00 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:00.210995 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:00.201054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:38855)
0928 16:38:00.201652 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:00.201670 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:00.201842 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:00.202221 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:00.202227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:00.202756 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:00.202969 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:00.202981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.204108 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.204112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:00.205255 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:00.205274 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.206051 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.206057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:00.206256 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:00.206945 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:00.206965 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:00.210595 (+ 3630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":307,"threads_started":1}
W20260928 16:38:00.211265 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:00.201078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:38855 (local address 127.10.247.254:38351)
0928 16:38:00.201317 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:00.201325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:00.201769 (+ 444us) server_negotiation.cc:407] Connection header received
0928 16:38:00.201977 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:00.201982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:00.202057 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:00.202154 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:00.203141 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.203951 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.205411 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.205889 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.206011 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:00.210953 (+ 4942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:38855: BlockingRecv error: recv got EOF from 127.10.247.193:38855 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 28 16:38:00 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:00 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:00.266124 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:00.256331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:45143)
0928 16:38:00.257046 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:00.257065 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:00.257238 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:00.257643 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:00.257649 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:00.258150 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:00.258369 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:00.258383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.259485 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.259490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:00.260445 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:00.260461 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.261365 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.261372 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:00.261555 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:00.262240 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:00.262262 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:00.265755 (+ 3493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":443,"threads_started":1}
W20260928 16:38:00.266266 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:00.256486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:45143 (local address 127.10.247.254:38351)
0928 16:38:00.256760 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:00.256767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:00.257221 (+ 454us) server_negotiation.cc:407] Connection header received
0928 16:38:00.257383 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:00.257387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:00.257480 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:00.257573 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:00.258546 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.259338 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.260600 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.261190 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.261312 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:00.266006 (+ 4694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:45143: BlockingRecv error: recv got EOF from 127.10.247.195:45143 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 28 16:38:00 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:00 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:00.738870 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:00.729131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:48833)
0928 16:38:00.729767 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:00.729785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:00.729923 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:00.730340 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:00.730345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:00.730869 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:00.731090 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:00.731104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.732231 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.732236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:00.733275 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:00.733293 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.734078 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.734083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:00.734246 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:00.734964 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:00.734989 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:00.738537 (+ 3548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":373,"threads_started":1}
W20260928 16:38:00.739063 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:00.729191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:48833 (local address 127.10.247.254:38351)
0928 16:38:00.729375 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:00.729380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:00.729858 (+ 478us) server_negotiation.cc:407] Connection header received
0928 16:38:00.730058 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:00.730063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:00.730146 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:00.730238 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:00.731273 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.732054 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.733416 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:00.733886 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:00.733989 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:00.738819 (+ 4830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:48833: BlockingRecv error: recv got EOF from 127.10.247.194:48833 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 28 16:38:01 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:01 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:01.223078 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:01.212588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:55639)
0928 16:38:01.213388 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:01.213408 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:01.213575 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:01.213971 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:01.213974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:01.214445 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:01.214661 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:01.214674 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.215988 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.215994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:01.217343 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:01.217364 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.218369 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.218375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:01.218528 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:01.219355 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:01.219376 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:01.222683 (+ 3307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":421,"threads_started":1}
W20260928 16:38:01.223516 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:01.212801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:55639 (local address 127.10.247.254:38351)
0928 16:38:01.213054 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:01.213060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:01.213541 (+ 481us) server_negotiation.cc:407] Connection header received
0928 16:38:01.213715 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:01.213722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:01.213826 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:01.213935 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:01.214827 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.215783 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.217523 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.218152 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.218265 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:01.223005 (+ 4740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:55639: BlockingRecv error: recv got EOF from 127.10.247.193:55639 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Sep 28 16:38:01 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:01 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:01.276940 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:01.267351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:54471)
0928 16:38:01.267948 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:01.267967 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:01.268118 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:01.268483 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:01.268486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:01.269017 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:01.269214 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:01.269226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.270484 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.270489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:01.271684 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:01.271701 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.272441 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.272445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:01.272590 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:01.273330 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:01.273350 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:01.276581 (+ 3231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":337,"threads_started":1}
W20260928 16:38:01.277245 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:01.267477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:54471 (local address 127.10.247.254:38351)
0928 16:38:01.267758 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:01.267766 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:01.268057 (+ 291us) server_negotiation.cc:407] Connection header received
0928 16:38:01.268238 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:01.268242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:01.268348 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:01.268443 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:01.269397 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.270323 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.271826 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.272300 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.272404 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:01.276926 (+ 4522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:54471: BlockingRecv error: recv got EOF from 127.10.247.195:54471 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 28 16:38:01 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:01 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:01.749886 20571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:01.740292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:44047)
0928 16:38:01.740966 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:01.740985 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:01.741176 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:01.741566 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:01.741571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:01.742097 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:01.742298 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:01.742310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.743440 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.743445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:01.744458 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:01.744473 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.745289 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.745295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:01.745493 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:01.746184 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:01.746206 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:01.749498 (+ 3292us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":351,"threads_started":1}
W20260928 16:38:01.750105 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:01.740470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:44047 (local address 127.10.247.254:38351)
0928 16:38:01.740726 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:01.740734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:01.741072 (+ 338us) server_negotiation.cc:407] Connection header received
0928 16:38:01.741312 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:01.741316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:01.741408 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:01.741496 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:01.742456 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.743275 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.744643 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:01.745129 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:01.745251 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:01.749809 (+ 4558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:44047: BlockingRecv error: recv got EOF from 127.10.247.194:44047 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 28 16:38:02 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:02 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:02.234923 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:02.224701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:60369)
0928 16:38:02.225288 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:02.225309 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:02.225510 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:02.225914 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:02.225920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:02.226526 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:02.226825 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:02.226840 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.228071 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.228077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:02.229139 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:02.229153 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.229934 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.229940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:02.230106 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:02.230889 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:02.230921 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:02.234521 (+ 3600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":302,"threads_started":1}
W20260928 16:38:02.235189 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:02.224849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:60369 (local address 127.10.247.254:38351)
0928 16:38:02.225096 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:02.225103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:02.225416 (+ 313us) server_negotiation.cc:407] Connection header received
0928 16:38:02.225668 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:02.225673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:02.225757 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:02.225844 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:02.227005 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.227915 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.229292 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.229764 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.229908 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:02.234877 (+ 4969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:60369: BlockingRecv error: recv got EOF from 127.10.247.193:60369 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Sep 28 16:38:02 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:02 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:02.288339 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:02.278373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:53193)
0928 16:38:02.279174 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:02.279193 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:02.279368 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:02.279783 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:02.279787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:02.280292 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:02.280506 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:02.280518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.281718 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.281723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:02.282788 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:02.282804 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.283647 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.283652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:02.283794 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:02.284470 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:02.284491 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:02.287901 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":432,"threads_started":1}
W20260928 16:38:02.288513 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:02.278474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:53193 (local address 127.10.247.254:38351)
0928 16:38:02.278863 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:02.278871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:02.279302 (+ 431us) server_negotiation.cc:407] Connection header received
0928 16:38:02.279513 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:02.279518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:02.279635 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:02.279720 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:02.280654 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.281544 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.282952 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.283442 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.283548 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:02.288232 (+ 4684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:53193: BlockingRecv error: recv got EOF from 127.10.247.195:53193 (error 108)
Metrics: {"server-negotiator.queue_time_us":203}
Sep 28 16:38:02 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:02 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:02.765708 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:02.751275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:52805)
0928 16:38:02.751904 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:02.751927 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:02.752117 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:02.752739 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:02.752742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:02.753194 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:02.753435 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:02.753445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.754584 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.754590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:02.755620 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:02.755635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.756434 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.756440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:02.756673 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:02.761446 (+ 4773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:02.761474 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:02.765354 (+ 3880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":300,"threads_started":1}
W20260928 16:38:02.765846 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:02.751441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:52805 (local address 127.10.247.254:38351)
0928 16:38:02.751771 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:02.751778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:02.752010 (+ 232us) server_negotiation.cc:407] Connection header received
0928 16:38:02.752290 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:02.752296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:02.752391 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:02.752468 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:02.753601 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.754433 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.755770 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:02.756256 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:02.756381 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:02.765646 (+ 9265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:52805: BlockingRecv error: recv got EOF from 127.10.247.194:52805 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Sep 28 16:38:03 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:03 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.193@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:03.247150 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:03.236758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.193:55331)
0928 16:38:03.237541 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:03.237559 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:03.237740 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:03.238133 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:03.238138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:03.238676 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:03.238882 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:03.238893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.240058 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.240062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:03.241142 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:03.241156 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.241991 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.242002 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:03.242191 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:03.242884 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:03.242905 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:03.246742 (+ 3837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":402,"threads_started":1}
W20260928 16:38:03.247351 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:03.236925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:55331 (local address 127.10.247.254:38351)
0928 16:38:03.237180 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:03.237188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:03.237683 (+ 495us) server_negotiation.cc:407] Connection header received
0928 16:38:03.237882 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:03.237887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:03.237968 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:03.238075 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:03.239075 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.239900 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.241297 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.241815 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.241944 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:03.247078 (+ 5134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.193:55331: BlockingRecv error: recv got EOF from 127.10.247.193:55331 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 28 16:38:03 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:03 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.195@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:03.299494 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:03.289748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.195:52707)
0928 16:38:03.290380 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:03.290399 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:03.290565 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:03.290993 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:03.290998 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:03.291479 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:03.291734 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:03.291747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.292967 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.292973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:03.293995 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:03.294010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.294827 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.294833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:03.295026 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:03.295739 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:03.295759 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:03.299207 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":334,"threads_started":1}
W20260928 16:38:03.299659 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:03.289927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:52707 (local address 127.10.247.254:38351)
0928 16:38:03.290232 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:03.290239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:03.290496 (+ 257us) server_negotiation.cc:407] Connection header received
0928 16:38:03.290730 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:03.290735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:03.290837 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:03.290930 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:03.291907 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.292817 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.294174 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.294673 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.294798 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:03.299423 (+ 4625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.195:52707: BlockingRecv error: recv got EOF from 127.10.247.195:52707 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 28 16:38:03 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 16:38:03 dist-test-slave-9fn6 krb5kdc[19521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.247.194@KRBTEST.COM for oryx/127.10.247.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 16:38:03.777510 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:03.767383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:38351 (local address 127.10.247.194:51441)
0928 16:38:03.768109 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:03.768133 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:03.768331 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:03.768853 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:03.768858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:03.769508 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:03.769786 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:03.769801 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.771193 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.771197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:03.772201 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:03.772214 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.773094 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.773099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:03.773239 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:03.773890 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:03.773909 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:03.777200 (+ 3291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.247.254:38351: Server oryx/127.10.247.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":362,"threads_started":1}
W20260928 16:38:03.777734 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:03.767561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:51441 (local address 127.10.247.254:38351)
0928 16:38:03.767877 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:03.767885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:03.768245 (+ 360us) server_negotiation.cc:407] Connection header received
0928 16:38:03.768543 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:03.768550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:03.768677 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:03.768791 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:03.769980 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.771037 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.772355 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:03.772934 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:03.773051 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:03.777460 (+ 4409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.247.194:51441: BlockingRecv error: recv got EOF from 127.10.247.194:51441 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
I20260928 16:38:04.127806 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20115
I20260928 16:38:04.143630 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:04.129406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.10.247.193:41355)
0928 16:38:04.130178 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:04.130189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:04.130233 (+ 44us) server_negotiation.cc:407] Connection header received
0928 16:38:04.130316 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:04.130324 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:04.130637 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:04.130866 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:04.132599 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:04.133921 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:04.135550 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:04.136335 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:04.136471 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:04.139701 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:04.139713 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:04.139733 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:04.139788 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:04.142866 (+ 3078us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.247.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0928 16:38:04.143317 (+ 451us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40308: Request ticket server kudu/127.10.247.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":369,"threads_started":1}
W20260928 16:38:04.143778 20579 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40308: Request ticket server kudu/127.10.247.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260928 16:38:04.143898 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20115: Not authorized: Client connection negotiation failed: client connection to 127.10.247.193:41355: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.247.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260928 16:38:04.144179 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 20115
I20260928 16:38:04.163573 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20267
I20260928 16:38:04.179018 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:04.165232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53812 (local address 127.10.247.194:43425)
0928 16:38:04.166068 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:04.166077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:04.166119 (+ 42us) server_negotiation.cc:407] Connection header received
0928 16:38:04.166198 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:04.166206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:04.166512 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:04.166750 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:04.168411 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:04.169677 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:04.171072 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:04.171726 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:04.171859 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:04.174945 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:04.174956 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:04.174977 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:04.175036 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:04.178145 (+ 3109us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.247.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0928 16:38:04.178602 (+ 457us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53812: Request ticket server kudu/127.10.247.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":395,"threads_started":1}
W20260928 16:38:04.179173 20580 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53812: Request ticket server kudu/127.10.247.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260928 16:38:04.179195 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20267: Not authorized: Client connection negotiation failed: client connection to 127.10.247.194:43425: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.247.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260928 16:38:04.179381 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 20267
I20260928 16:38:04.197603 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20399
I20260928 16:38:04.212905 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:04.198947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.10.247.195:40027)
0928 16:38:04.199661 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:04.199670 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:04.199703 (+ 33us) server_negotiation.cc:407] Connection header received
0928 16:38:04.199786 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:04.199794 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:04.200104 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:04.200324 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:04.202080 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:04.203289 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:04.205090 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:04.205847 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:04.205982 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:04.209266 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:04.209278 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:04.209298 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:04.209356 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:04.212226 (+ 2870us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.247.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0928 16:38:04.212587 (+ 361us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47804: Request ticket server kudu/127.10.247.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":339,"threads_started":1}
W20260928 16:38:04.213029 20581 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47804: Request ticket server kudu/127.10.247.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260928 16:38:04.213105 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20399: Not authorized: Client connection negotiation failed: client connection to 127.10.247.195:40027: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.247.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260928 16:38:04.213351 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 20399
I20260928 16:38:04.229952 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20045
I20260928 16:38:04.231567 20108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:04.370024 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 20045
2026-09-28T16:38:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21922 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-28T16:38:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:38:04Z Disabled control of system clock
I20260928 16:38:04.457444 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:36141
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:36141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 16:38:04.848428 20586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:04.849000 20586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:04.849131 20586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:04.857031 20586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:38:04.857152 20586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:04.857197 20586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:04.857257 20586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:38:04.857306 20586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:38:04.867197 20586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:36141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:36141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:04.869017 20586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:04.871460 20586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:04.884720 20592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:04.884743 20594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:04.884791 20591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:04.885943 20586 server_base.cc:1034] running on GCE node
I20260928 16:38:04.886911 20586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:04.888690 20586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:04.889907 20586 hybrid_clock.cc:648] HybridClock initialized: now 1790613484889857 us; error 33 us; skew 500 ppm
I20260928 16:38:04.890306 20586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:38:04.892872 20586 webserver.cc:460] Webserver started at http://127.10.247.254:45537/ using document root <none> and password file <none>
I20260928 16:38:04.893713 20586 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:04.893812 20586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:04.894181 20586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:04.897706 20586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "c41950e847b44853b746c66586294f6b"
format_stamp: "Formatted at 2026-09-28 16:38:04 on dist-test-slave-9fn6"
server_key: "4fe7339afd51d775bb13bbb845f34dae"
server_key_iv: "a885d6266618dbbca050d4ec0bd3e7f8"
server_key_version: "encryptionkey@0"
I20260928 16:38:04.898658 20586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "c41950e847b44853b746c66586294f6b"
format_stamp: "Formatted at 2026-09-28 16:38:04 on dist-test-slave-9fn6"
server_key: "4fe7339afd51d775bb13bbb845f34dae"
server_key_iv: "a885d6266618dbbca050d4ec0bd3e7f8"
server_key_version: "encryptionkey@0"
I20260928 16:38:04.904573 20586 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.006s
I20260928 16:38:04.908538 20600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:04.910259 20586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:38:04.910476 20586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "c41950e847b44853b746c66586294f6b"
format_stamp: "Formatted at 2026-09-28 16:38:04 on dist-test-slave-9fn6"
server_key: "4fe7339afd51d775bb13bbb845f34dae"
server_key_iv: "a885d6266618dbbca050d4ec0bd3e7f8"
server_key_version: "encryptionkey@0"
I20260928 16:38:04.910722 20586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:04.934499 20586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:04.935668 20586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:04.935997 20586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:04.956290 20586 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:36141
I20260928 16:38:04.956321 20652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:36141 every 8 connection(s)
I20260928 16:38:04.958030 20586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:38:04.962911 20653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:38:04.966768 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 20586
I20260928 16:38:04.966982 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:38:04.967693 11231 external_mini_cluster.cc:1428] Setting key 65cd19b0d77bfd5f913991926fd96784
I20260928 16:38:04.977777 20653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b: Bootstrap starting.
I20260928 16:38:04.982259 20653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b: Neither blocks nor log segments found. Creating new log.
I20260928 16:38:04.983661 20656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:04.970125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.10.247.254:36141)
0928 16:38:04.971282 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:04.971295 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:04.971348 (+ 53us) server_negotiation.cc:407] Connection header received
0928 16:38:04.973243 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:04.973275 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:04.973969 (+ 694us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:04.974398 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:04.975254 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:04.976700 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:04.978619 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:04.979368 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:04.979518 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:04.980962 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:04.980970 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:04.980987 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:04.981629 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:04.981734 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:04.981763 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:04.981858 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:04.982118 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 16:38:04.982486 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":425,"threads_started":1}
I20260928 16:38:04.983670 20653 log.cc:824] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:38:04.987331 20653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b: No bootstrap required, opened a new log
I20260928 16:38:04.996153 20653 raft_consensus.cc:348] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41950e847b44853b746c66586294f6b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36141 } }
I20260928 16:38:04.996737 20653 raft_consensus.cc:374] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:38:04.996851 20653 raft_consensus.cc:729] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c41950e847b44853b746c66586294f6b, State: Initialized, Role: FOLLOWER
I20260928 16:38:04.997560 20653 consensus_queue.cc:260] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41950e847b44853b746c66586294f6b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36141 } }
I20260928 16:38:04.997833 20653 raft_consensus.cc:388] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:38:04.998025 20653 raft_consensus.cc:482] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:38:04.998229 20653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:38:05.000737 20653 raft_consensus.cc:504] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41950e847b44853b746c66586294f6b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36141 } }
I20260928 16:38:05.001360 20653 leader_election.cc:302] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c41950e847b44853b746c66586294f6b; no voters:
I20260928 16:38:05.002108 20653 leader_election.cc:288] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:38:05.002295 20658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:38:05.003026 20658 raft_consensus.cc:686] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [term 1 LEADER]: Becoming Leader. State: Replica: c41950e847b44853b746c66586294f6b, State: Running, Role: LEADER
I20260928 16:38:05.003693 20658 consensus_queue.cc:237] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41950e847b44853b746c66586294f6b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36141 } }
I20260928 16:38:05.004597 20653 sys_catalog.cc:564] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:38:05.006829 20659 sys_catalog.cc:455] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c41950e847b44853b746c66586294f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41950e847b44853b746c66586294f6b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36141 } } }
I20260928 16:38:05.006933 20660 sys_catalog.cc:455] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c41950e847b44853b746c66586294f6b. Latest consensus state: current_term: 1 leader_uuid: "c41950e847b44853b746c66586294f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41950e847b44853b746c66586294f6b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36141 } } }
I20260928 16:38:05.007272 20659 sys_catalog.cc:458] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [sys.catalog]: This master's current role is: LEADER
I20260928 16:38:05.007302 20660 sys_catalog.cc:458] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b [sys.catalog]: This master's current role is: LEADER
I20260928 16:38:05.009490 20664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:38:05.015893 20664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:38:05.027119 20664 catalog_manager.cc:1347] Generated new cluster ID: b8b5edb9f3f24c39affb85afcd258a83
I20260928 16:38:05.027267 20664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:38:05.048630 20664 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:38:05.050274 20664 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:38:05.060428 20664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c41950e847b44853b746c66586294f6b: Generated new TSK 0
I20260928 16:38:05.061578 20664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:38:05.075098 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36141
--builtin_ntp_servers=127.10.247.212:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 16:38:05.472971 20677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:05.473325 20677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:05.473459 20677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:05.481104 20677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:05.481211 20677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:05.481352 20677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:38:05.491107 20677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:05.492762 20677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:05.494913 20677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:05.509922 20682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:05.509969 20683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:05.510000 20685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:05.510691 20677 server_base.cc:1034] running on GCE node
I20260928 16:38:05.511451 20677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:05.512669 20677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:05.513892 20677 hybrid_clock.cc:648] HybridClock initialized: now 1790613485513838 us; error 45 us; skew 500 ppm
I20260928 16:38:05.514294 20677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:38:05.517129 20677 webserver.cc:460] Webserver started at http://127.10.247.193:35267/ using document root <none> and password file <none>
I20260928 16:38:05.517967 20677 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:05.518072 20677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:05.518441 20677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:05.521931 20677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "55534e47e585422ca4e7952d147e28cf"
format_stamp: "Formatted at 2026-09-28 16:38:05 on dist-test-slave-9fn6"
server_key: "d5a60e1cf2a2e1ea767ead6202ac153b"
server_key_iv: "3ca7627a557ca6cf504c2926f34a4e48"
server_key_version: "encryptionkey@0"
I20260928 16:38:05.522852 20677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "55534e47e585422ca4e7952d147e28cf"
format_stamp: "Formatted at 2026-09-28 16:38:05 on dist-test-slave-9fn6"
server_key: "d5a60e1cf2a2e1ea767ead6202ac153b"
server_key_iv: "3ca7627a557ca6cf504c2926f34a4e48"
server_key_version: "encryptionkey@0"
I20260928 16:38:05.529173 20677 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:38:05.533253 20691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:05.534904 20677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:38:05.535140 20677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "55534e47e585422ca4e7952d147e28cf"
format_stamp: "Formatted at 2026-09-28 16:38:05 on dist-test-slave-9fn6"
server_key: "d5a60e1cf2a2e1ea767ead6202ac153b"
server_key_iv: "3ca7627a557ca6cf504c2926f34a4e48"
server_key_version: "encryptionkey@0"
I20260928 16:38:05.535368 20677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:05.552559 20677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:05.553719 20677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:05.554075 20677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:05.555197 20677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:38:05.557024 20677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:38:05.557125 20677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:05.557238 20677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:38:05.557309 20677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:05.590938 20677 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:35591
I20260928 16:38:05.590973 20804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:35591 every 8 connection(s)
I20260928 16:38:05.592483 20677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:38:05.599740 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 20677
I20260928 16:38:05.599927 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:38:05.600841 11231 external_mini_cluster.cc:1428] Setting key ff8c2436d888cbc05c54874828863f11
I20260928 16:38:05.604480 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36141
--builtin_ntp_servers=127.10.247.212:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 16:38:05.608297 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:05.595504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:57589 (local address 127.10.247.254:36141)
0928 16:38:05.596009 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:05.596015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:05.597275 (+ 1260us) server_negotiation.cc:407] Connection header received
0928 16:38:05.599938 (+ 2663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:05.599945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:05.600068 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:05.600186 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:05.601800 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:05.602812 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:05.605186 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:05.605707 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:05.605835 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:05.607505 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:05.607510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:05.607513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:05.607534 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:05.607590 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:05.607603 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:05.607670 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:05.607892 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 16:38:05.607991 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":256,"threads_started":1}
I20260928 16:38:05.609877 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:05.596045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:36141 (local address 127.10.247.193:57589)
0928 16:38:05.597074 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:05.597141 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:05.598927 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:05.600494 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:05.600509 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:05.600677 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:05.601603 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:05.601629 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:05.603006 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:05.603012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:05.605031 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:05.605047 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:05.605965 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:05.605973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:05.606293 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:05.607104 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:38:05.607107 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:05.607174 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:38:05.607290 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:38:05.607682 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:38:05.607692 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:38:05.607699 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 16:38:05.607878 (+ 179us) client_negotiation.cc:240] Negotiation successful
0928 16:38:05.608319 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":292,"threads_started":1}
I20260928 16:38:05.611642 20805 heartbeater.cc:347] Connected to a master server at 127.10.247.254:36141
I20260928 16:38:05.611974 20805 heartbeater.cc:464] Registering TS with master...
I20260928 16:38:05.612953 20805 heartbeater.cc:511] Master 127.10.247.254:36141 requested a full tablet report, sending...
I20260928 16:38:05.615417 20617 ts_manager.cc:194] Registered new tserver with Master: 55534e47e585422ca4e7952d147e28cf (127.10.247.193:35591)
I20260928 16:38:05.617759 20617 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.193:57589
W20260928 16:38:05.990080 20809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:05.990412 20809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:05.990521 20809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:05.998329 20809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:05.998466 20809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:05.998588 20809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:38:06.008630 20809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:06.010385 20809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:06.012573 20809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:06.027873 20814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:06.027799 20817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:06.027873 20815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:06.028519 20809 server_base.cc:1034] running on GCE node
I20260928 16:38:06.029304 20809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:06.030438 20809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:06.031677 20809 hybrid_clock.cc:648] HybridClock initialized: now 1790613486031587 us; error 56 us; skew 500 ppm
I20260928 16:38:06.032037 20809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:38:06.034685 20809 webserver.cc:460] Webserver started at http://127.10.247.194:41623/ using document root <none> and password file <none>
I20260928 16:38:06.035682 20809 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:06.035812 20809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:06.036224 20809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:06.039746 20809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "e08ff2c1436541fd8fcd96695c6fb3b0"
format_stamp: "Formatted at 2026-09-28 16:38:06 on dist-test-slave-9fn6"
server_key: "fefbdb9532827199a31896b078b6d5a4"
server_key_iv: "536177bf3648b8167daccf1a8984d310"
server_key_version: "encryptionkey@0"
I20260928 16:38:06.040596 20809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "e08ff2c1436541fd8fcd96695c6fb3b0"
format_stamp: "Formatted at 2026-09-28 16:38:06 on dist-test-slave-9fn6"
server_key: "fefbdb9532827199a31896b078b6d5a4"
server_key_iv: "536177bf3648b8167daccf1a8984d310"
server_key_version: "encryptionkey@0"
I20260928 16:38:06.046730 20809 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260928 16:38:06.050887 20823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:06.052531 20809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:38:06.052771 20809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "e08ff2c1436541fd8fcd96695c6fb3b0"
format_stamp: "Formatted at 2026-09-28 16:38:06 on dist-test-slave-9fn6"
server_key: "fefbdb9532827199a31896b078b6d5a4"
server_key_iv: "536177bf3648b8167daccf1a8984d310"
server_key_version: "encryptionkey@0"
I20260928 16:38:06.053013 20809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:06.083612 20809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:06.084739 20809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:06.085070 20809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:06.086191 20809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:38:06.087891 20809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:38:06.087993 20809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:06.088110 20809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:38:06.088191 20809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:06.121534 20809 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:43515
I20260928 16:38:06.121608 20936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:43515 every 8 connection(s)
I20260928 16:38:06.123174 20809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:38:06.129596 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 20809
I20260928 16:38:06.129812 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:38:06.130438 11231 external_mini_cluster.cc:1428] Setting key d4d1f1bf18a85bb38932bc9a529cff8e
I20260928 16:38:06.133736 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36141
--builtin_ntp_servers=127.10.247.212:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 16:38:06.138535 20939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:06.126262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:46787 (local address 127.10.247.254:36141)
0928 16:38:06.126824 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:06.126831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:06.128300 (+ 1469us) server_negotiation.cc:407] Connection header received
0928 16:38:06.130826 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:06.130831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:06.130920 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:06.131028 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:06.132297 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.133228 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.135526 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.135967 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.136097 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:06.137722 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:06.137726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:06.137730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:06.137751 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:06.137798 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:06.137811 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:06.137881 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:06.138141 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 16:38:06.138241 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":246,"threads_started":1}
I20260928 16:38:06.140048 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:06.126957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:36141 (local address 127.10.247.194:46787)
0928 16:38:06.128144 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:06.128185 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:06.130483 (+ 2298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:06.131308 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:06.131317 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:06.131424 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:06.132125 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:06.132146 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.133387 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.133393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:06.135372 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:06.135391 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.136205 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.136211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:06.136512 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:06.137341 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:38:06.137344 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:06.137404 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:38:06.137512 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:38:06.137892 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:38:06.137903 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:38:06.137910 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 16:38:06.138113 (+ 203us) client_negotiation.cc:240] Negotiation successful
0928 16:38:06.138559 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":329,"threads_started":1}
I20260928 16:38:06.141626 20937 heartbeater.cc:347] Connected to a master server at 127.10.247.254:36141
I20260928 16:38:06.141968 20937 heartbeater.cc:464] Registering TS with master...
I20260928 16:38:06.142872 20937 heartbeater.cc:511] Master 127.10.247.254:36141 requested a full tablet report, sending...
I20260928 16:38:06.144554 20617 ts_manager.cc:194] Registered new tserver with Master: e08ff2c1436541fd8fcd96695c6fb3b0 (127.10.247.194:43515)
I20260928 16:38:06.145713 20617 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.194:46787
W20260928 16:38:06.514123 20941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:06.514465 20941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:06.514559 20941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:06.522470 20941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:06.522603 20941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:06.522718 20941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:38:06.533387 20941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:06.534996 20941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:06.537211 20941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:06.551666 20946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:06.551614 20947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:06.551637 20949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:06.554167 20941 server_base.cc:1034] running on GCE node
I20260928 16:38:06.554917 20941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:06.556035 20941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:06.557248 20941 hybrid_clock.cc:648] HybridClock initialized: now 1790613486557178 us; error 61 us; skew 500 ppm
I20260928 16:38:06.557585 20941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:38:06.560101 20941 webserver.cc:460] Webserver started at http://127.10.247.195:41739/ using document root <none> and password file <none>
I20260928 16:38:06.560948 20941 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:06.561064 20941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:06.561405 20941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:06.564666 20941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "fa749206acb64c148b05cce5a537f2bc"
format_stamp: "Formatted at 2026-09-28 16:38:06 on dist-test-slave-9fn6"
server_key: "dfe135208ebd30721f65d63bea8da7bc"
server_key_iv: "910289444502fb874d064d1155514e01"
server_key_version: "encryptionkey@0"
I20260928 16:38:06.565512 20941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "fa749206acb64c148b05cce5a537f2bc"
format_stamp: "Formatted at 2026-09-28 16:38:06 on dist-test-slave-9fn6"
server_key: "dfe135208ebd30721f65d63bea8da7bc"
server_key_iv: "910289444502fb874d064d1155514e01"
server_key_version: "encryptionkey@0"
I20260928 16:38:06.571060 20941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:38:06.574816 20955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:06.576329 20941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:38:06.576494 20941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "fa749206acb64c148b05cce5a537f2bc"
format_stamp: "Formatted at 2026-09-28 16:38:06 on dist-test-slave-9fn6"
server_key: "dfe135208ebd30721f65d63bea8da7bc"
server_key_iv: "910289444502fb874d064d1155514e01"
server_key_version: "encryptionkey@0"
I20260928 16:38:06.576762 20941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:06.592031 20941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:06.593015 20941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:06.593331 20941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:06.594444 20941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:38:06.596184 20941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:38:06.596285 20941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:06.596402 20941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:38:06.596475 20941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:06.622437 20805 heartbeater.cc:503] Master 127.10.247.254:36141 was elected leader, sending a full tablet report...
I20260928 16:38:06.630856 20941 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:38389
I20260928 16:38:06.630892 21068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:38389 every 8 connection(s)
I20260928 16:38:06.632416 20941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:38:06.634783 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 20941
I20260928 16:38:06.634971 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:38:06.635643 11231 external_mini_cluster.cc:1428] Setting key f5cb1f0aa4971a58354ffc11c0a78d96
I20260928 16:38:06.646607 20939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:06.635526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:45673 (local address 127.10.247.254:36141)
0928 16:38:06.635770 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:06.635778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:06.637698 (+ 1920us) server_negotiation.cc:407] Connection header received
0928 16:38:06.639729 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:06.639736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:06.639829 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:06.639920 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:06.641249 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.642065 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.643720 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.644145 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.644257 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:06.645717 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:06.645723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:06.645726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:06.645747 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:06.645785 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:06.645797 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:06.645874 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:06.646123 (+ 249us) server_negotiation.cc:299] Negotiation successful
0928 16:38:06.646247 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 16:38:06.647785 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:06.636253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:36141 (local address 127.10.247.195:45673)
0928 16:38:06.637532 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:06.637588 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:06.639431 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:06.640213 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:06.640223 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:06.640321 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:06.641078 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:06.641098 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.642218 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.642222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:06.643578 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:06.643591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.644368 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.644375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:06.644660 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:06.645367 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:38:06.645370 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:06.645415 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:38:06.645525 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:38:06.645884 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:38:06.645895 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:38:06.645904 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 16:38:06.646082 (+ 178us) client_negotiation.cc:240] Negotiation successful
0928 16:38:06.646561 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":427,"threads_started":1}
I20260928 16:38:06.649300 21069 heartbeater.cc:347] Connected to a master server at 127.10.247.254:36141
I20260928 16:38:06.649888 21069 heartbeater.cc:464] Registering TS with master...
I20260928 16:38:06.651051 21069 heartbeater.cc:511] Master 127.10.247.254:36141 requested a full tablet report, sending...
I20260928 16:38:06.652689 20617 ts_manager.cc:194] Registered new tserver with Master: fa749206acb64c148b05cce5a537f2bc (127.10.247.195:38389)
I20260928 16:38:06.653610 20617 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.195:45673
I20260928 16:38:06.663810 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 16:38:06.670339 20939 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:06.668772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.10.247.254:36141)
0928 16:38:06.669003 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:06.669009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:06.669317 (+ 308us) server_negotiation.cc:407] Connection header received
0928 16:38:06.669464 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:06.669468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:06.669538 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:06.669626 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:06.670062 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55572: BlockingRecv error: recv got EOF from 127.0.0.1:55572 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:38:06.670749 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20677
I20260928 16:38:06.678853 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:06.671827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33562 (local address 127.10.247.193:35591)
0928 16:38:06.672401 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:06.672409 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:06.672439 (+ 30us) server_negotiation.cc:407] Connection header received
0928 16:38:06.672501 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:06.672508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:06.672824 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:06.672960 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:06.673382 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.674486 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.675721 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.676329 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.676433 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:06.677458 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:06.677466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:06.677482 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:06.678144 (+ 662us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:06.678200 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:06.678220 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:06.678319 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:06.678395 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 16:38:06.678528 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":267,"threads_started":1}
I20260928 16:38:06.680267 20800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:06.768549 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 20677
I20260928 16:38:06.784942 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20809
I20260928 16:38:06.793799 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:06.785966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50134 (local address 127.10.247.194:43515)
0928 16:38:06.786536 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:06.786555 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:06.786586 (+ 31us) server_negotiation.cc:407] Connection header received
0928 16:38:06.786659 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:06.786667 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:06.786946 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:06.787123 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:06.787600 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.788713 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.790034 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.790652 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.790784 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:06.791944 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:06.791954 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:06.791971 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:06.792853 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:06.792914 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:06.792939 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:06.793072 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:06.793215 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 16:38:06.793386 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":256,"threads_started":1}
I20260928 16:38:06.795389 20932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:06.884335 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 20809
I20260928 16:38:06.901616 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20941
I20260928 16:38:06.911432 21082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:06.902975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.10.247.195:38389)
0928 16:38:06.903741 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:06.903749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:06.903786 (+ 37us) server_negotiation.cc:407] Connection header received
0928 16:38:06.903873 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:06.903882 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:06.904191 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:06.904376 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:06.904973 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.906197 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.907562 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:06.908235 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:06.908357 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:06.909638 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:06.909649 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:06.909668 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:06.910413 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:06.910476 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:06.910507 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:06.910630 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:06.910796 (+ 166us) server_negotiation.cc:299] Negotiation successful
0928 16:38:06.910996 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":400,"threads_started":1}
I20260928 16:38:06.913139 21064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:07.010206 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 20941
I20260928 16:38:07.026518 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 20586
I20260928 16:38:07.028038 20648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:07.145201 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 20586
2026-09-28T16:38:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2770 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-28T16:38:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:38:07Z Disabled control of system clock
I20260928 16:38:07.229028 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44993
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:44993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 16:38:07.613983 21088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:07.614380 21088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:07.614476 21088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:07.622722 21088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:38:07.622896 21088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:07.622943 21088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:07.623010 21088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:38:07.623056 21088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:38:07.633721 21088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:44993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:07.635524 21088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:07.637939 21088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:07.652066 21093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:07.652128 21096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:07.652107 21094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:07.652958 21088 server_base.cc:1034] running on GCE node
I20260928 16:38:07.654050 21088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:07.655861 21088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:07.657191 21088 hybrid_clock.cc:648] HybridClock initialized: now 1790613487657106 us; error 76 us; skew 500 ppm
I20260928 16:38:07.657618 21088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:38:07.660297 21088 webserver.cc:460] Webserver started at http://127.10.247.254:37881/ using document root <none> and password file <none>
I20260928 16:38:07.661199 21088 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:07.661303 21088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:07.661665 21088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:07.665266 21088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "8c55b6c8fe1b477dbe704e337f575429"
format_stamp: "Formatted at 2026-09-28 16:38:07 on dist-test-slave-9fn6"
server_key: "217b32acfbc6127a1b74a64c83297e3a"
server_key_iv: "77d2201822b9e5150eb2948d1a53f747"
server_key_version: "encryptionkey@0"
I20260928 16:38:07.666227 21088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "8c55b6c8fe1b477dbe704e337f575429"
format_stamp: "Formatted at 2026-09-28 16:38:07 on dist-test-slave-9fn6"
server_key: "217b32acfbc6127a1b74a64c83297e3a"
server_key_iv: "77d2201822b9e5150eb2948d1a53f747"
server_key_version: "encryptionkey@0"
I20260928 16:38:07.672593 21088 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:38:07.677080 21102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:07.678803 21088 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:38:07.679032 21088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "8c55b6c8fe1b477dbe704e337f575429"
format_stamp: "Formatted at 2026-09-28 16:38:07 on dist-test-slave-9fn6"
server_key: "217b32acfbc6127a1b74a64c83297e3a"
server_key_iv: "77d2201822b9e5150eb2948d1a53f747"
server_key_version: "encryptionkey@0"
I20260928 16:38:07.679297 21088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:07.686424 21088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:07.687381 21088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:07.687722 21088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:07.707370 21088 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:44993
I20260928 16:38:07.707396 21153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:44993 every 8 connection(s)
I20260928 16:38:07.709218 21088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:38:07.714267 21154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:38:07.718521 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 21088
I20260928 16:38:07.718765 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:38:07.719458 11231 external_mini_cluster.cc:1428] Setting key 0b511886d1ec3850315e8c66a9035410
I20260928 16:38:07.730685 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:07.722083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.10.247.254:44993)
0928 16:38:07.723392 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:07.723423 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:07.723488 (+ 65us) server_negotiation.cc:407] Connection header received
0928 16:38:07.725553 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:07.725600 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:07.726354 (+ 754us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:07.726941 (+ 587us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:07.728207 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:07.728216 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:07.728236 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:07.728255 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:07.728342 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:07.728366 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:07.728641 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:07.728783 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 16:38:07.729134 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":397,"threads_started":1}
I20260928 16:38:07.730939 21154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429: Bootstrap starting.
I20260928 16:38:07.735374 21154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429: Neither blocks nor log segments found. Creating new log.
I20260928 16:38:07.736589 21154 log.cc:824] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429: Log is configured to *not* fsync() on all Append() calls
I20260928 16:38:07.740259 21154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429: No bootstrap required, opened a new log
I20260928 16:38:07.748968 21154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c55b6c8fe1b477dbe704e337f575429" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44993 } }
I20260928 16:38:07.749482 21154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:38:07.749589 21154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c55b6c8fe1b477dbe704e337f575429, State: Initialized, Role: FOLLOWER
I20260928 16:38:07.750298 21154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c55b6c8fe1b477dbe704e337f575429" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44993 } }
I20260928 16:38:07.750554 21154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:38:07.750700 21154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:38:07.750873 21154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:38:07.753708 21154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c55b6c8fe1b477dbe704e337f575429" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44993 } }
I20260928 16:38:07.754313 21154 leader_election.cc:302] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c55b6c8fe1b477dbe704e337f575429; no voters:
I20260928 16:38:07.755021 21154 leader_election.cc:288] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:38:07.755183 21159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:38:07.755910 21159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [term 1 LEADER]: Becoming Leader. State: Replica: 8c55b6c8fe1b477dbe704e337f575429, State: Running, Role: LEADER
I20260928 16:38:07.756603 21159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c55b6c8fe1b477dbe704e337f575429" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44993 } }
I20260928 16:38:07.757511 21154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:38:07.759675 21161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c55b6c8fe1b477dbe704e337f575429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c55b6c8fe1b477dbe704e337f575429" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44993 } } }
I20260928 16:38:07.760114 21161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [sys.catalog]: This master's current role is: LEADER
I20260928 16:38:07.761270 21162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c55b6c8fe1b477dbe704e337f575429. Latest consensus state: current_term: 1 leader_uuid: "8c55b6c8fe1b477dbe704e337f575429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c55b6c8fe1b477dbe704e337f575429" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44993 } } }
I20260928 16:38:07.761691 21162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429 [sys.catalog]: This master's current role is: LEADER
I20260928 16:38:07.763782 21167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:38:07.769361 21167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:38:07.779798 21167 catalog_manager.cc:1347] Generated new cluster ID: a70ed81ec449495ea551fe1f9e778143
I20260928 16:38:07.779950 21167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:38:07.790591 21167 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:38:07.791111 21167 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:38:07.798839 21167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c55b6c8fe1b477dbe704e337f575429: Generated new TSK 0
I20260928 16:38:07.800392 21167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:38:07.817131 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44993
--builtin_ntp_servers=127.10.247.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 16:38:08.212574 21179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:08.213039 21179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:08.213171 21179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:08.221202 21179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:08.221352 21179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:08.221517 21179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:38:08.231804 21179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:08.233593 21179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:08.235877 21179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:08.251482 21184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:08.251449 21185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:08.251420 21187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:08.252632 21179 server_base.cc:1034] running on GCE node
I20260928 16:38:08.253373 21179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:08.254587 21179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:08.255862 21179 hybrid_clock.cc:648] HybridClock initialized: now 1790613488255797 us; error 51 us; skew 500 ppm
I20260928 16:38:08.256260 21179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:38:08.264977 21179 webserver.cc:460] Webserver started at http://127.10.247.193:42213/ using document root <none> and password file <none>
I20260928 16:38:08.265954 21179 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:08.266079 21179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:08.266445 21179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:08.270093 21179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "4386d42326ec4908b613828bb3ee73db"
format_stamp: "Formatted at 2026-09-28 16:38:08 on dist-test-slave-9fn6"
server_key: "d9c81ced3e0cce21831e4ceeec79e08b"
server_key_iv: "a902fdee8569de805ac460339f380ef1"
server_key_version: "encryptionkey@0"
I20260928 16:38:08.271023 21179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "4386d42326ec4908b613828bb3ee73db"
format_stamp: "Formatted at 2026-09-28 16:38:08 on dist-test-slave-9fn6"
server_key: "d9c81ced3e0cce21831e4ceeec79e08b"
server_key_iv: "a902fdee8569de805ac460339f380ef1"
server_key_version: "encryptionkey@0"
I20260928 16:38:08.277509 21179 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260928 16:38:08.281725 21193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:08.283453 21179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260928 16:38:08.283666 21179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "4386d42326ec4908b613828bb3ee73db"
format_stamp: "Formatted at 2026-09-28 16:38:08 on dist-test-slave-9fn6"
server_key: "d9c81ced3e0cce21831e4ceeec79e08b"
server_key_iv: "a902fdee8569de805ac460339f380ef1"
server_key_version: "encryptionkey@0"
I20260928 16:38:08.283921 21179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:08.290671 21179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:08.291678 21179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:08.291965 21179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:08.293077 21179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:38:08.294766 21179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:38:08.294871 21179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:08.294987 21179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:38:08.295060 21179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:08.329038 21179 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:37809
I20260928 16:38:08.329077 21306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:37809 every 8 connection(s)
I20260928 16:38:08.330601 21179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:38:08.337276 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 21179
I20260928 16:38:08.337515 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:38:08.338340 11231 external_mini_cluster.cc:1428] Setting key f3e236c71426e40ba93466c4c653caa1
I20260928 16:38:08.340157 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:08.333615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:57859 (local address 127.10.247.254:44993)
0928 16:38:08.334188 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:08.334197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:08.335549 (+ 1352us) server_negotiation.cc:407] Connection header received
0928 16:38:08.337560 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:08.337568 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:08.337685 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:08.337834 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:08.339320 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:08.339325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:08.339328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:08.339349 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:08.339402 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:08.339416 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:08.339542 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:08.339725 (+ 183us) server_negotiation.cc:299] Negotiation successful
0928 16:38:08.339803 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":249,"threads_started":1}
I20260928 16:38:08.341280 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:08.334311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44993 (local address 127.10.247.193:57859)
0928 16:38:08.335363 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:08.335407 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:08.337237 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:08.338064 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:08.338075 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:08.338193 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:08.338970 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:38:08.338973 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:08.339022 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:38:08.339125 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:38:08.339515 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:38:08.339524 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:38:08.339531 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 16:38:08.339780 (+ 249us) client_negotiation.cc:240] Negotiation successful
0928 16:38:08.340035 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":303,"threads_started":1}
I20260928 16:38:08.341802 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44993
--builtin_ntp_servers=127.10.247.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 16:38:08.342923 21307 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44993
I20260928 16:38:08.343237 21307 heartbeater.cc:464] Registering TS with master...
I20260928 16:38:08.343931 21307 heartbeater.cc:511] Master 127.10.247.254:44993 requested a full tablet report, sending...
I20260928 16:38:08.346470 21119 ts_manager.cc:194] Registered new tserver with Master: 4386d42326ec4908b613828bb3ee73db (127.10.247.193:37809)
W20260928 16:38:08.715731 21311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:08.716094 21311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:08.716230 21311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:08.724246 21311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:08.724385 21311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:08.724534 21311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:38:08.734750 21311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:08.736517 21311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:08.738884 21311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:08.753502 21316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:08.753423 21317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:08.753516 21319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:08.754118 21311 server_base.cc:1034] running on GCE node
I20260928 16:38:08.754801 21311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:08.755906 21311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:08.757113 21311 hybrid_clock.cc:648] HybridClock initialized: now 1790613488757056 us; error 55 us; skew 500 ppm
I20260928 16:38:08.757476 21311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:38:08.760042 21311 webserver.cc:460] Webserver started at http://127.10.247.194:40015/ using document root <none> and password file <none>
I20260928 16:38:08.760921 21311 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:08.761029 21311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:08.761380 21311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:08.764966 21311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "7adc39bd45374b09b3e981989003c510"
format_stamp: "Formatted at 2026-09-28 16:38:08 on dist-test-slave-9fn6"
server_key: "aeb1f621afb5fd0d75351d9436c2663f"
server_key_iv: "7c1177b3eb586db09457c0244f9de2a5"
server_key_version: "encryptionkey@0"
I20260928 16:38:08.765851 21311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "7adc39bd45374b09b3e981989003c510"
format_stamp: "Formatted at 2026-09-28 16:38:08 on dist-test-slave-9fn6"
server_key: "aeb1f621afb5fd0d75351d9436c2663f"
server_key_iv: "7c1177b3eb586db09457c0244f9de2a5"
server_key_version: "encryptionkey@0"
I20260928 16:38:08.771806 21311 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:38:08.775568 21325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:08.777079 21311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:38:08.777242 21311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "7adc39bd45374b09b3e981989003c510"
format_stamp: "Formatted at 2026-09-28 16:38:08 on dist-test-slave-9fn6"
server_key: "aeb1f621afb5fd0d75351d9436c2663f"
server_key_iv: "7c1177b3eb586db09457c0244f9de2a5"
server_key_version: "encryptionkey@0"
I20260928 16:38:08.777472 21311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:08.783715 21311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:08.784660 21311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:08.784927 21311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:08.785910 21311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:38:08.787454 21311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:38:08.787559 21311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:08.787683 21311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:38:08.787770 21311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:08.819680 21311 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:36897
I20260928 16:38:08.819721 21438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:36897 every 8 connection(s)
I20260928 16:38:08.821280 21311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:38:08.822259 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 21311
I20260928 16:38:08.822480 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:38:08.823143 11231 external_mini_cluster.cc:1428] Setting key 849bdc0b859fd7275f1f37be1ce84c15
I20260928 16:38:08.827244 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44993
--builtin_ntp_servers=127.10.247.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 16:38:08.832216 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:08.824729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:50499 (local address 127.10.247.254:44993)
0928 16:38:08.824917 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:08.824922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:08.827028 (+ 2106us) server_negotiation.cc:407] Connection header received
0928 16:38:08.829675 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:08.829680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:08.829744 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:08.829826 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:08.831503 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:08.831507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:08.831510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:08.831532 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:08.831574 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:08.831588 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:08.831717 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:08.831888 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 16:38:08.831958 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:38:08.833906 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:08.825484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44993 (local address 127.10.247.194:50499)
0928 16:38:08.826839 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:08.826892 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:08.829295 (+ 2403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:08.830155 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:08.830168 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:08.830314 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:08.831101 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:38:08.831104 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:08.831170 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:38:08.831299 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:38:08.831675 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:38:08.831685 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:38:08.831694 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 16:38:08.831992 (+ 298us) client_negotiation.cc:240] Negotiation successful
0928 16:38:08.832316 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":373,"threads_started":1}
I20260928 16:38:08.835489 21439 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44993
I20260928 16:38:08.835865 21439 heartbeater.cc:464] Registering TS with master...
I20260928 16:38:08.836756 21439 heartbeater.cc:511] Master 127.10.247.254:44993 requested a full tablet report, sending...
I20260928 16:38:08.838562 21119 ts_manager.cc:194] Registered new tserver with Master: 7adc39bd45374b09b3e981989003c510 (127.10.247.194:36897)
W20260928 16:38:09.229490 21442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:09.229842 21442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:09.230005 21442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:09.237957 21442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:09.238106 21442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:09.238272 21442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:38:09.248368 21442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:44993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:09.250216 21442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:09.252486 21442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:09.268074 21447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:09.268061 21448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:09.267994 21450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:09.269006 21442 server_base.cc:1034] running on GCE node
I20260928 16:38:09.269742 21442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:09.270932 21442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:09.272169 21442 hybrid_clock.cc:648] HybridClock initialized: now 1790613489272107 us; error 56 us; skew 500 ppm
I20260928 16:38:09.272534 21442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:38:09.275282 21442 webserver.cc:460] Webserver started at http://127.10.247.195:46539/ using document root <none> and password file <none>
I20260928 16:38:09.276109 21442 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:09.276222 21442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:09.276597 21442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:09.280081 21442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "4aceb170a6f0438a8bd26f0e8fb5b69e"
format_stamp: "Formatted at 2026-09-28 16:38:09 on dist-test-slave-9fn6"
server_key: "e74e49887d5b716abb445b4f4a7ae219"
server_key_iv: "3dc3df30d2efb2388b4bbccd976e85c8"
server_key_version: "encryptionkey@0"
I20260928 16:38:09.281023 21442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "4aceb170a6f0438a8bd26f0e8fb5b69e"
format_stamp: "Formatted at 2026-09-28 16:38:09 on dist-test-slave-9fn6"
server_key: "e74e49887d5b716abb445b4f4a7ae219"
server_key_iv: "3dc3df30d2efb2388b4bbccd976e85c8"
server_key_version: "encryptionkey@0"
I20260928 16:38:09.287405 21442 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260928 16:38:09.291484 21456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:09.293212 21442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:38:09.293376 21442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "4aceb170a6f0438a8bd26f0e8fb5b69e"
format_stamp: "Formatted at 2026-09-28 16:38:09 on dist-test-slave-9fn6"
server_key: "e74e49887d5b716abb445b4f4a7ae219"
server_key_iv: "3dc3df30d2efb2388b4bbccd976e85c8"
server_key_version: "encryptionkey@0"
I20260928 16:38:09.293613 21442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:09.300275 21442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:09.301342 21442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:09.301641 21442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:09.302745 21442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:38:09.304447 21442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:38:09.304557 21442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:09.304747 21442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:38:09.304826 21442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:09.338841 21442 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:45513
I20260928 16:38:09.338886 21569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:45513 every 8 connection(s)
I20260928 16:38:09.340507 21442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:38:09.341254 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 21442
I20260928 16:38:09.341493 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:38:09.342103 11231 external_mini_cluster.cc:1428] Setting key cd6463a257715b40916e71656050c833
I20260928 16:38:09.350054 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:09.343494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:50407 (local address 127.10.247.254:44993)
0928 16:38:09.344013 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:09.344020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:09.345500 (+ 1480us) server_negotiation.cc:407] Connection header received
0928 16:38:09.347501 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:09.347506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:09.347587 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:09.347693 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:09.349179 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:09.349186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:09.349191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:09.349213 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:09.349263 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:09.349276 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:09.349384 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:09.349569 (+ 185us) server_negotiation.cc:299] Negotiation successful
0928 16:38:09.349663 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":269,"threads_started":1}
I20260928 16:38:09.350275 21307 heartbeater.cc:503] Master 127.10.247.254:44993 was elected leader, sending a full tablet report...
I20260928 16:38:09.351227 21573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:09.344230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44993 (local address 127.10.247.195:50407)
0928 16:38:09.345349 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:09.345388 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:09.347210 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:09.347970 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:09.347981 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:09.348088 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:09.348860 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:38:09.348863 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:09.348907 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:38:09.349007 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:38:09.349364 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:38:09.349374 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:38:09.349382 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 16:38:09.349656 (+ 274us) client_negotiation.cc:240] Negotiation successful
0928 16:38:09.349978 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":365,"threads_started":1}
I20260928 16:38:09.352763 21570 heartbeater.cc:347] Connected to a master server at 127.10.247.254:44993
I20260928 16:38:09.353084 21570 heartbeater.cc:464] Registering TS with master...
I20260928 16:38:09.353807 21570 heartbeater.cc:511] Master 127.10.247.254:44993 requested a full tablet report, sending...
I20260928 16:38:09.355312 21119 ts_manager.cc:194] Registered new tserver with Master: 4aceb170a6f0438a8bd26f0e8fb5b69e (127.10.247.195:45513)
I20260928 16:38:09.358856 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 16:38:09.365815 21572 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:38:09.364076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.10.247.254:44993)
0928 16:38:09.364275 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:09.364280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:09.364878 (+ 598us) server_negotiation.cc:407] Connection header received
0928 16:38:09.365062 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:09.365067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:09.365133 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:09.365229 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:09.365541 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45690: BlockingRecv error: recv got EOF from 127.0.0.1:45690 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:38:09.366953 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 21179
I20260928 16:38:09.371027 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:09.368099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.10.247.193:37809)
0928 16:38:09.368749 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:09.368767 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:09.368799 (+ 32us) server_negotiation.cc:407] Connection header received
0928 16:38:09.368872 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:09.368881 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:09.369160 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:09.369314 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:09.370313 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:09.370321 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:09.370334 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:09.370346 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:09.370395 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:09.370411 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:09.370555 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:09.370611 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 16:38:09.370716 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":285,"threads_started":1}
I20260928 16:38:09.372548 21302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:09.458880 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 21179
I20260928 16:38:09.475586 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 21311
I20260928 16:38:09.479982 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:09.476886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47384 (local address 127.10.247.194:36897)
0928 16:38:09.477646 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:09.477657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:09.477692 (+ 35us) server_negotiation.cc:407] Connection header received
0928 16:38:09.477774 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:09.477781 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:09.478070 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:09.478217 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:09.479219 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:09.479226 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:09.479239 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:09.479252 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:09.479306 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:09.479321 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:09.479445 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:09.479497 (+ 52us) server_negotiation.cc:299] Negotiation successful
0928 16:38:09.479668 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":288,"threads_started":1}
I20260928 16:38:09.481567 21434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:09.568094 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 21311
I20260928 16:38:09.583985 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 21442
I20260928 16:38:09.588438 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:09.585203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50304 (local address 127.10.247.195:45513)
0928 16:38:09.585929 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:09.585937 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:09.585975 (+ 38us) server_negotiation.cc:407] Connection header received
0928 16:38:09.586055 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:09.586063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:09.586356 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:09.586503 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:09.587706 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:09.587714 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:09.587727 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:38:09.587745 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:09.587792 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:38:09.587809 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:09.587951 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:09.588008 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 16:38:09.588106 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":344,"threads_started":1}
I20260928 16:38:09.589972 21565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:09.677604 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 21442
I20260928 16:38:09.693490 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 21088
I20260928 16:38:09.695075 21149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:09.810566 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 21088
2026-09-28T16:38:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2665 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:38:09 dist-test-slave-9fn6 krb5kdc[21589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:38:09 dist-test-slave-9fn6 krb5kdc[21589](info): set up 2 sockets
Sep 28 16:38:09 dist-test-slave-9fn6 krb5kdc[21589](info): commencing operation
krb5kdc: starting...
W20260928 16:38:10.869874 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:38:11 dist-test-slave-9fn6 krb5kdc[21589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:38:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:38:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:38:11.162411 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:35653
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:35653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:38:11.551590 21602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:11.552014 21602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:11.552158 21602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:11.560143 21602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:38:11.560312 21602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:11.560395 21602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:11.560482 21602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:38:11.560541 21602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:38:11.571492 21602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:35653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:35653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:11.573346 21602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:11.574994 21602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:11.588738 21607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:11.588711 21610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:11.588672 21608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:11.589720 21602 server_base.cc:1034] running on GCE node
I20260928 16:38:11.590636 21602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:11.592434 21602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:11.593658 21602 hybrid_clock.cc:648] HybridClock initialized: now 1790613491593587 us; error 58 us; skew 500 ppm
I20260928 16:38:11.594043 21602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:38:11 dist-test-slave-9fn6 krb5kdc[21589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613491, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:38:11.597568 21602 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:38:11.599370 21602 webserver.cc:460] Webserver started at http://127.10.247.254:45491/ using document root <none> and password file <none>
I20260928 16:38:11.600215 21602 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:11.600338 21602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:11.600749 21602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:11.604370 21602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "9b448bae7e964005b1517e57b3e70615"
format_stamp: "Formatted at 2026-09-28 16:38:11 on dist-test-slave-9fn6"
server_key: "ac008ce426158313141ab7a17eabcf7e"
server_key_iv: "06649891a5f6f52dbe3f2f774b48bbff"
server_key_version: "encryptionkey@0"
I20260928 16:38:11.605280 21602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "9b448bae7e964005b1517e57b3e70615"
format_stamp: "Formatted at 2026-09-28 16:38:11 on dist-test-slave-9fn6"
server_key: "ac008ce426158313141ab7a17eabcf7e"
server_key_iv: "06649891a5f6f52dbe3f2f774b48bbff"
server_key_version: "encryptionkey@0"
I20260928 16:38:11.611495 21602 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:38:11.615466 21617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:11.617259 21602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:38:11.617460 21602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "9b448bae7e964005b1517e57b3e70615"
format_stamp: "Formatted at 2026-09-28 16:38:11 on dist-test-slave-9fn6"
server_key: "ac008ce426158313141ab7a17eabcf7e"
server_key_iv: "06649891a5f6f52dbe3f2f774b48bbff"
server_key_version: "encryptionkey@0"
I20260928 16:38:11.617705 21602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:11.639861 21602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:11.644915 21602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:11.645247 21602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:11.665869 21602 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:35653
I20260928 16:38:11.665896 21668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:35653 every 8 connection(s)
I20260928 16:38:11.667513 21602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:38:11.672076 21669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:38:11.675043 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 21602
I20260928 16:38:11.675292 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:38:11.676044 11231 external_mini_cluster.cc:1428] Setting key 862aa6ce0c3fa9393e309d8b5481e554
I20260928 16:38:11.686023 21669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615: Bootstrap starting.
I20260928 16:38:11.689769 21669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615: Neither blocks nor log segments found. Creating new log.
I20260928 16:38:11.690964 21669 log.cc:824] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615: Log is configured to *not* fsync() on all Append() calls
Sep 28 16:38:11 dist-test-slave-9fn6 krb5kdc[21589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:38:11.695077 21669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615: No bootstrap required, opened a new log
I20260928 16:38:11.702239 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:11.678630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.10.247.254:35653)
0928 16:38:11.679482 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:11.679496 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:11.679543 (+ 47us) server_negotiation.cc:407] Connection header received
0928 16:38:11.681463 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:11.681490 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:11.681988 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:11.682378 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:11.684481 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:11.686054 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:11.687463 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:11.688230 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:11.688417 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:11.693048 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:11.693062 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:11.693085 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:11.693143 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:11.697045 (+ 3902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:11.697903 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:11.697911 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:11.697920 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:11.698144 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:11.698583 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:11.698588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:11.698590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:11.699273 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:11.699643 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:11.700175 (+ 532us) server_negotiation.cc:299] Negotiation successful
0928 16:38:11.700802 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":366,"threads_started":1}
I20260928 16:38:11.704169 21669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b448bae7e964005b1517e57b3e70615" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 35653 } }
I20260928 16:38:11.704855 21669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:38:11.704983 21669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b448bae7e964005b1517e57b3e70615, State: Initialized, Role: FOLLOWER
I20260928 16:38:11.705802 21669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b448bae7e964005b1517e57b3e70615" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 35653 } }
I20260928 16:38:11.706122 21669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:38:11.706307 21669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:38:11.706514 21669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:38:11.709116 21669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b448bae7e964005b1517e57b3e70615" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 35653 } }
I20260928 16:38:11.709731 21669 leader_election.cc:302] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b448bae7e964005b1517e57b3e70615; no voters:
I20260928 16:38:11.710389 21669 leader_election.cc:288] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:38:11.710544 21674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:38:11.711297 21674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [term 1 LEADER]: Becoming Leader. State: Replica: 9b448bae7e964005b1517e57b3e70615, State: Running, Role: LEADER
I20260928 16:38:11.711927 21674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b448bae7e964005b1517e57b3e70615" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 35653 } }
I20260928 16:38:11.712930 21669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:38:11.715441 21676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b448bae7e964005b1517e57b3e70615. Latest consensus state: current_term: 1 leader_uuid: "9b448bae7e964005b1517e57b3e70615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b448bae7e964005b1517e57b3e70615" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 35653 } } }
I20260928 16:38:11.715986 21676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [sys.catalog]: This master's current role is: LEADER
I20260928 16:38:11.715739 21675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b448bae7e964005b1517e57b3e70615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b448bae7e964005b1517e57b3e70615" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 35653 } } }
I20260928 16:38:11.716221 21675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615 [sys.catalog]: This master's current role is: LEADER
I20260928 16:38:11.717547 21681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:38:11.724421 21681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:38:11.736286 21681 catalog_manager.cc:1347] Generated new cluster ID: a845be4cafb1426c9443bf5a1cc1bc3e
I20260928 16:38:11.736454 21681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:38:11.763804 21681 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:38:11.765882 21681 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:38:11.777604 21681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615: Generated new TSK 0
I20260928 16:38:11.778852 21681 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:38:11.865522 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:35653
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:38:12.276307 21695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:12.276834 21695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:12.277025 21695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:12.285590 21695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:12.285748 21695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:12.285892 21695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:38:12.297083 21695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:35653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:12.298807 21695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:12.300537 21695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:12.316232 21701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:12.316275 21704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:12.316299 21702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:12.317683 21695 server_base.cc:1034] running on GCE node
I20260928 16:38:12.318521 21695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:12.319810 21695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:12.321041 21695 hybrid_clock.cc:648] HybridClock initialized: now 1790613492320958 us; error 63 us; skew 500 ppm
I20260928 16:38:12.321624 21695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:38:12 dist-test-slave-9fn6 krb5kdc[21589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613492, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:38:12.325703 21695 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:38:12.327629 21695 webserver.cc:460] Webserver started at http://127.10.247.193:39987/ using document root <none> and password file <none>
I20260928 16:38:12.328523 21695 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:12.328706 21695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:12.329053 21695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:12.332593 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "deaf8b17f4054bd9a7e2712b0a28dc64"
format_stamp: "Formatted at 2026-09-28 16:38:12 on dist-test-slave-9fn6"
server_key: "a6a950b0f1d8bd53262bfcd5503b2b30"
server_key_iv: "c2ef4cf342ee54f9000e83148ad3dc69"
server_key_version: "encryptionkey@0"
I20260928 16:38:12.333527 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "deaf8b17f4054bd9a7e2712b0a28dc64"
format_stamp: "Formatted at 2026-09-28 16:38:12 on dist-test-slave-9fn6"
server_key: "a6a950b0f1d8bd53262bfcd5503b2b30"
server_key_iv: "c2ef4cf342ee54f9000e83148ad3dc69"
server_key_version: "encryptionkey@0"
I20260928 16:38:12.339531 21695 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260928 16:38:12.343894 21711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:12.345803 21695 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260928 16:38:12.346014 21695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "deaf8b17f4054bd9a7e2712b0a28dc64"
format_stamp: "Formatted at 2026-09-28 16:38:12 on dist-test-slave-9fn6"
server_key: "a6a950b0f1d8bd53262bfcd5503b2b30"
server_key_iv: "c2ef4cf342ee54f9000e83148ad3dc69"
server_key_version: "encryptionkey@0"
I20260928 16:38:12.346269 21695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:12.369907 21695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:12.375253 21695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:12.375604 21695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:12.376852 21695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:38:12.378526 21695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:38:12.378623 21695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:12.378762 21695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:38:12.378818 21695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:12.412992 21695 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:38695
I20260928 16:38:12.413043 21823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:38695 every 8 connection(s)
I20260928 16:38:12.414602 21695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:38:12.422940 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 21695
I20260928 16:38:12.423233 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:38:12.424149 11231 external_mini_cluster.cc:1428] Setting key 8c837a9adbf297790c01d6ff7a11011a
Sep 28 16:38:12 dist-test-slave-9fn6 krb5kdc[21589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613492, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:38:12.439028 21826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:12.417598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:37565 (local address 127.10.247.254:35653)
0928 16:38:12.418242 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:12.418250 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:12.419372 (+ 1122us) server_negotiation.cc:407] Connection header received
0928 16:38:12.421516 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:12.421523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:12.421627 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:12.421742 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:12.424046 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:12.425002 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:12.427100 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:12.427532 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:12.427642 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:12.432185 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:12.432192 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:12.432197 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:12.432241 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:12.435436 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:12.436334 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:12.436339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:12.436342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:12.436502 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:12.436980 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:12.436985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:12.436988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:12.437376 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:12.437525 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:12.438485 (+ 960us) server_negotiation.cc:299] Negotiation successful
0928 16:38:12.438714 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":266,"threads_started":1}
I20260928 16:38:12.440721 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:12.418255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:35653 (local address 127.10.247.193:37565)
0928 16:38:12.419158 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:12.419205 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:12.421162 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:12.421998 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:12.422009 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:12.422834 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:12.423835 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:12.423866 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:12.425197 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:12.425202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:12.426930 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:12.426946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:12.427796 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:12.427803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:12.428129 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:12.429026 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:12.429069 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:12.431901 (+ 2832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:38:12.435629 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:38:12.435639 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:38:12.435657 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:38:12.436208 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:38:12.436625 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:38:12.436630 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:38:12.436633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:38:12.436876 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:38:12.437490 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:38:12.437496 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:38:12.437999 (+ 503us) client_negotiation.cc:769] Sending connection context
0928 16:38:12.438355 (+ 356us) client_negotiation.cc:240] Negotiation successful
0928 16:38:12.439051 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":301,"threads_started":1}
I20260928 16:38:12.442575 21824 heartbeater.cc:347] Connected to a master server at 127.10.247.254:35653
I20260928 16:38:12.442970 21824 heartbeater.cc:464] Registering TS with master...
I20260928 16:38:12.444032 21824 heartbeater.cc:511] Master 127.10.247.254:35653 requested a full tablet report, sending...
I20260928 16:38:12.447255 21634 ts_manager.cc:194] Registered new tserver with Master: deaf8b17f4054bd9a7e2712b0a28dc64 (127.10.247.193:38695)
I20260928 16:38:12.449527 21634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:37565
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:38:12.492836 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:35653
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:38:12.895819 21830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:12.896216 21830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:12.896354 21830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:12.904568 21830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:12.904779 21830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:12.904968 21830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:38:12.916297 21830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:35653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:12.918202 21830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:12.920074 21830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:12.935173 21838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:12.935410 21835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:12.935418 21836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:12.936486 21830 server_base.cc:1034] running on GCE node
I20260928 16:38:12.937592 21830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:12.938728 21830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:12.939982 21830 hybrid_clock.cc:648] HybridClock initialized: now 1790613492939911 us; error 39 us; skew 500 ppm
I20260928 16:38:12.940341 21830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:38:12 dist-test-slave-9fn6 krb5kdc[21589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613492, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:38:12.943971 21830 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:38:12.945729 21830 webserver.cc:460] Webserver started at http://127.10.247.194:44421/ using document root <none> and password file <none>
I20260928 16:38:12.946552 21830 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:12.946681 21830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:12.947023 21830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:12.950995 21830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "164c354707c841caa5fdcbce00c89078"
format_stamp: "Formatted at 2026-09-28 16:38:12 on dist-test-slave-9fn6"
server_key: "afed78c5215804b5569abfbc9b4da699"
server_key_iv: "6fb44419ab0cca6ca96f19fb16c6efa3"
server_key_version: "encryptionkey@0"
I20260928 16:38:12.952050 21830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "164c354707c841caa5fdcbce00c89078"
format_stamp: "Formatted at 2026-09-28 16:38:12 on dist-test-slave-9fn6"
server_key: "afed78c5215804b5569abfbc9b4da699"
server_key_iv: "6fb44419ab0cca6ca96f19fb16c6efa3"
server_key_version: "encryptionkey@0"
I20260928 16:38:12.958932 21830 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260928 16:38:12.963055 21845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:12.964663 21830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:38:12.964843 21830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "164c354707c841caa5fdcbce00c89078"
format_stamp: "Formatted at 2026-09-28 16:38:12 on dist-test-slave-9fn6"
server_key: "afed78c5215804b5569abfbc9b4da699"
server_key_iv: "6fb44419ab0cca6ca96f19fb16c6efa3"
server_key_version: "encryptionkey@0"
I20260928 16:38:12.965067 21830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:12.988679 21830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:12.993959 21830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:12.994341 21830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:12.995558 21830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:38:12.997308 21830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:38:12.997401 21830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:12.997532 21830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:38:12.997586 21830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:13.032235 21830 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:46153
I20260928 16:38:13.032279 21957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:46153 every 8 connection(s)
I20260928 16:38:13.033762 21830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:38:13.043826 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 21830
I20260928 16:38:13.044055 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:38:13.044569 11231 external_mini_cluster.cc:1428] Setting key 85c752ef0b722e9f7cb09596b1678cb3
Sep 28 16:38:13 dist-test-slave-9fn6 krb5kdc[21589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613492, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:38:13.058127 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.036825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:45147 (local address 127.10.247.254:35653)
0928 16:38:13.037341 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:13.037346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:13.038463 (+ 1117us) server_negotiation.cc:407] Connection header received
0928 16:38:13.040573 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:13.040580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:13.040706 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:13.040818 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:13.042713 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.043532 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.045572 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.046209 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.046327 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.050809 (+ 4482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:13.050815 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:13.050819 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:13.050868 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:13.054105 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:13.055152 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:13.055157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:13.055160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:13.055314 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:13.055798 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:13.055802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:13.055805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:13.056171 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:13.056306 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:13.057408 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0928 16:38:13.057705 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":226,"threads_started":1}
I20260928 16:38:13.059212 21961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.037468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:35653 (local address 127.10.247.194:45147)
0928 16:38:13.038274 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:13.038331 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:13.040253 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:13.041124 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:13.041134 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:13.041774 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:13.042488 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.042509 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.043705 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.043710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.045382 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.045400 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.046486 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.046494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.046839 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.047692 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:13.047721 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:13.050521 (+ 2800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:38:13.054305 (+ 3784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:38:13.054317 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:38:13.054342 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:38:13.055031 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:38:13.055408 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:38:13.055413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:38:13.055416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:38:13.055699 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:38:13.056286 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:38:13.056296 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:38:13.056918 (+ 622us) client_negotiation.cc:769] Sending connection context
0928 16:38:13.057235 (+ 317us) client_negotiation.cc:240] Negotiation successful
0928 16:38:13.057770 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":298,"threads_started":1}
I20260928 16:38:13.061012 21958 heartbeater.cc:347] Connected to a master server at 127.10.247.254:35653
I20260928 16:38:13.061378 21958 heartbeater.cc:464] Registering TS with master...
I20260928 16:38:13.062304 21958 heartbeater.cc:511] Master 127.10.247.254:35653 requested a full tablet report, sending...
I20260928 16:38:13.064107 21634 ts_manager.cc:194] Registered new tserver with Master: 164c354707c841caa5fdcbce00c89078 (127.10.247.194:46153)
I20260928 16:38:13.065141 21634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:45147
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:38:13.108049 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:35653
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:38:13.453753 21824 heartbeater.cc:503] Master 127.10.247.254:35653 was elected leader, sending a full tablet report...
W20260928 16:38:13.510080 21964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:38:13.510465 21964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:38:13.510602 21964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:38:13.518929 21964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:38:13.519081 21964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:38:13.519248 21964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:38:13.530741 21964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:35653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:38:13.532495 21964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:38:13.534243 21964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:38:13.550325 21972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:13.550253 21970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:38:13.550532 21969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:38:13.551375 21964 server_base.cc:1034] running on GCE node
I20260928 16:38:13.552415 21964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:38:13.553789 21964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:38:13.555027 21964 hybrid_clock.cc:648] HybridClock initialized: now 1790613493554962 us; error 50 us; skew 500 ppm
I20260928 16:38:13.555418 21964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:38:13 dist-test-slave-9fn6 krb5kdc[21589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613493, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:38:13.566121 21964 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:38:13.568060 21964 webserver.cc:460] Webserver started at http://127.10.247.195:40669/ using document root <none> and password file <none>
I20260928 16:38:13.569038 21964 fs_manager.cc:359] Metadata directory not provided
I20260928 16:38:13.569175 21964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:38:13.569597 21964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:38:13.573477 21964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "d2da7b827019472aaad5a34795567038"
format_stamp: "Formatted at 2026-09-28 16:38:13 on dist-test-slave-9fn6"
server_key: "635f1638bae3e0ea8186e5862181ce26"
server_key_iv: "805400b461471b67db0010de0610a4a0"
server_key_version: "encryptionkey@0"
I20260928 16:38:13.574456 21964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "d2da7b827019472aaad5a34795567038"
format_stamp: "Formatted at 2026-09-28 16:38:13 on dist-test-slave-9fn6"
server_key: "635f1638bae3e0ea8186e5862181ce26"
server_key_iv: "805400b461471b67db0010de0610a4a0"
server_key_version: "encryptionkey@0"
I20260928 16:38:13.581255 21964 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260928 16:38:13.586176 21979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:13.588020 21964 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:38:13.588219 21964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "d2da7b827019472aaad5a34795567038"
format_stamp: "Formatted at 2026-09-28 16:38:13 on dist-test-slave-9fn6"
server_key: "635f1638bae3e0ea8186e5862181ce26"
server_key_iv: "805400b461471b67db0010de0610a4a0"
server_key_version: "encryptionkey@0"
I20260928 16:38:13.588454 21964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:38:13.604336 21964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:38:13.609607 21964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:38:13.609948 21964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:38:13.611183 21964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:38:13.613173 21964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:38:13.613261 21964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:13.613377 21964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:38:13.613433 21964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:38:13.650121 21964 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:38351
I20260928 16:38:13.650131 22091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:38351 every 8 connection(s)
I20260928 16:38:13.651772 21964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:38:13.660117 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 21964
I20260928 16:38:13.660351 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:38:13.660892 11231 external_mini_cluster.cc:1428] Setting key 49753c1290c9cac0abaccfac0babe40c
Sep 28 16:38:13 dist-test-slave-9fn6 krb5kdc[21589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613493, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:38:13.677239 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.654956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:45411 (local address 127.10.247.254:35653)
0928 16:38:13.655508 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:13.655515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:13.656782 (+ 1267us) server_negotiation.cc:407] Connection header received
0928 16:38:13.658898 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:13.658906 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:13.659006 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:13.659128 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:13.661167 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.662275 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.664585 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.665107 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.665218 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.670101 (+ 4883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:13.670107 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:13.670111 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:13.670159 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:13.673236 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:13.674342 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:13.674347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:13.674350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:13.674509 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:13.674998 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:13.675001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:13.675003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:13.675333 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:13.675467 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:13.676500 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0928 16:38:13.676831 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":264,"threads_started":1}
I20260928 16:38:13.678597 22095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.655664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:35653 (local address 127.10.247.195:45411)
0928 16:38:13.656530 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:13.656584 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:13.658538 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:13.659432 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:13.659442 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:13.660083 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:38:13.660917 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.660942 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.662457 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.662472 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.664421 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.664438 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.665361 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.665370 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.665712 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.666549 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:38:13.666583 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:38:13.669827 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:38:13.673433 (+ 3606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:38:13.673446 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:38:13.673471 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:38:13.674198 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:38:13.674622 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:38:13.674626 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:38:13.674628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:38:13.674879 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:38:13.675466 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:38:13.675476 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:38:13.675932 (+ 456us) client_negotiation.cc:769] Sending connection context
0928 16:38:13.676291 (+ 359us) client_negotiation.cc:240] Negotiation successful
0928 16:38:13.676968 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":358,"threads_started":1}
I20260928 16:38:13.680202 22092 heartbeater.cc:347] Connected to a master server at 127.10.247.254:35653
I20260928 16:38:13.680590 22092 heartbeater.cc:464] Registering TS with master...
I20260928 16:38:13.681622 22092 heartbeater.cc:511] Master 127.10.247.254:35653 requested a full tablet report, sending...
I20260928 16:38:13.683537 21634 ts_manager.cc:194] Registered new tserver with Master: d2da7b827019472aaad5a34795567038 (127.10.247.195:38351)
I20260928 16:38:13.684528 21634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:45411
I20260928 16:38:13.690982 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:38:13.711884 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.696686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53260 (local address 127.10.247.254:35653)
0928 16:38:13.696963 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:13.696970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:13.697234 (+ 264us) server_negotiation.cc:407] Connection header received
0928 16:38:13.697435 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:13.697439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:13.697549 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:13.697637 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:13.699252 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.700122 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.701593 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.702077 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.702206 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.705063 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:13.705070 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:13.705074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:13.705129 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:13.708297 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:13.709485 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:13.709491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:13.709493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:13.709704 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:13.710153 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:13.710157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:13.710159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:13.710592 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:13.710837 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:13.711301 (+ 464us) server_negotiation.cc:299] Negotiation successful
0928 16:38:13.711542 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260928 16:38:13.718391 21634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:38:13.727028 21634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:38:13.769441 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.745756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.10.247.193:38695)
0928 16:38:13.757345 (+ 11589us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:13.757361 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:13.758283 (+ 922us) server_negotiation.cc:407] Connection header received
0928 16:38:13.759050 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:13.759061 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:13.759324 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:13.759554 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:38:13.761371 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.762969 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.766315 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.767823 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.768024 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.768221 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:13.768729 (+ 508us) server_negotiation.cc:299] Negotiation successful
0928 16:38:13.768980 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11260,"thread_start_us":11612,"threads_started":1}
I20260928 16:38:13.769522 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.745523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:38695 (local address 127.0.0.1:46388)
0928 16:38:13.758127 (+ 12604us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:13.758145 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:13.758516 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:13.759905 (+ 1389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:13.759911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:13.759965 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:38:13.761106 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.761135 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.763183 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.763191 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.766093 (+ 2902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.766108 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.768075 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.768083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.768346 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.768419 (+ 73us) client_negotiation.cc:769] Sending connection context
0928 16:38:13.768574 (+ 155us) client_negotiation.cc:240] Negotiation successful
0928 16:38:13.768979 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12473,"mutex_wait_us":866,"spinlock_wait_cycles":32896,"thread_start_us":12328,"threads_started":1}
I20260928 16:38:13.770310 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.759491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44316 (local address 127.10.247.195:38351)
0928 16:38:13.760946 (+ 1455us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:13.760956 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:13.761620 (+ 664us) server_negotiation.cc:407] Connection header received
0928 16:38:13.761776 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:13.761783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:13.762027 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:13.762244 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:38:13.763460 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.764805 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.766956 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.768129 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.768386 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.768575 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:13.769650 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0928 16:38:13.769865 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1229,"thread_start_us":990,"threads_started":1}
I20260928 16:38:13.770483 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.759361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:38351 (local address 127.0.0.1:44316)
0928 16:38:13.761476 (+ 2115us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:13.761494 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:13.761628 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:13.762581 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:13.762599 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:13.762636 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:38:13.763217 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.763231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.764959 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.764965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.766768 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.766784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.768785 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.768791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.768969 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.769003 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 16:38:13.769186 (+ 183us) client_negotiation.cc:240] Negotiation successful
0928 16:38:13.769345 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1992,"mutex_wait_us":1,"thread_start_us":1813,"threads_started":1}
I20260928 16:38:13.771489 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.745066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:46153 (local address 127.0.0.1:33812)
0928 16:38:13.758016 (+ 12950us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:13.758092 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:13.758516 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:13.759883 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:13.759894 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:13.759954 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:38:13.760663 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.760679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.762907 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.762913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.766091 (+ 3178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.766108 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.770608 (+ 4500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.770615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:13.770818 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.770853 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 16:38:13.770938 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 16:38:13.771097 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12559,"mutex_wait_us":174,"spinlock_wait_cycles":9344,"thread_start_us":13175,"threads_started":1}
I20260928 16:38:13.772287 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.745569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33812 (local address 127.10.247.194:46153)
0928 16:38:13.746377 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:13.746391 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:13.758293 (+ 11902us) server_negotiation.cc:407] Connection header received
0928 16:38:13.758868 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:13.758880 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:13.759228 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:13.759434 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:38:13.760914 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.762701 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.768368 (+ 5667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.770038 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.770222 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:13.770412 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:13.771770 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0928 16:38:13.771952 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":353,"threads_started":1}
I20260928 16:38:13.775739 22027 tablet_service.cc:1467] Processing CreateTablet for tablet 2aabc0ceae8d43119bdb3c443c24494c (DEFAULT_TABLE table=test-table [id=d9c4ae5c9e5249f98ca6b7555cf05bd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:38:13.775892 21759 tablet_service.cc:1467] Processing CreateTablet for tablet 2aabc0ceae8d43119bdb3c443c24494c (DEFAULT_TABLE table=test-table [id=d9c4ae5c9e5249f98ca6b7555cf05bd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:38:13.776646 21893 tablet_service.cc:1467] Processing CreateTablet for tablet 2aabc0ceae8d43119bdb3c443c24494c (DEFAULT_TABLE table=test-table [id=d9c4ae5c9e5249f98ca6b7555cf05bd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:38:13.777529 22027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aabc0ceae8d43119bdb3c443c24494c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:38:13.777552 21759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aabc0ceae8d43119bdb3c443c24494c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:38:13.778214 21893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aabc0ceae8d43119bdb3c443c24494c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:38:13.791985 22111 tablet_bootstrap.cc:492] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: Bootstrap starting.
I20260928 16:38:13.795082 22113 tablet_bootstrap.cc:492] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: Bootstrap starting.
I20260928 16:38:13.796059 22112 tablet_bootstrap.cc:492] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: Bootstrap starting.
I20260928 16:38:13.797109 22111 tablet_bootstrap.cc:654] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: Neither blocks nor log segments found. Creating new log.
I20260928 16:38:13.798842 22111 log.cc:824] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: Log is configured to *not* fsync() on all Append() calls
I20260928 16:38:13.800076 22113 tablet_bootstrap.cc:654] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: Neither blocks nor log segments found. Creating new log.
I20260928 16:38:13.800357 22112 tablet_bootstrap.cc:654] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: Neither blocks nor log segments found. Creating new log.
I20260928 16:38:13.801842 22113 log.cc:824] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: Log is configured to *not* fsync() on all Append() calls
I20260928 16:38:13.802044 22112 log.cc:824] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: Log is configured to *not* fsync() on all Append() calls
I20260928 16:38:13.805757 22111 tablet_bootstrap.cc:492] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: No bootstrap required, opened a new log
I20260928 16:38:13.805783 22113 tablet_bootstrap.cc:492] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: No bootstrap required, opened a new log
I20260928 16:38:13.806133 22111 ts_tablet_manager.cc:1397] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260928 16:38:13.806146 22113 ts_tablet_manager.cc:1397] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260928 16:38:13.806236 22112 tablet_bootstrap.cc:492] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: No bootstrap required, opened a new log
I20260928 16:38:13.806579 22112 ts_tablet_manager.cc:1397] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260928 16:38:13.815181 22111 raft_consensus.cc:348] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } }
I20260928 16:38:13.815786 22111 raft_consensus.cc:374] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:38:13.815923 22111 raft_consensus.cc:729] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2da7b827019472aaad5a34795567038, State: Initialized, Role: FOLLOWER
I20260928 16:38:13.816772 22111 consensus_queue.cc:260] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } }
I20260928 16:38:13.818333 22092 heartbeater.cc:503] Master 127.10.247.254:35653 was elected leader, sending a full tablet report...
I20260928 16:38:13.819074 22111 ts_tablet_manager.cc:1428] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260928 16:38:13.818476 22112 raft_consensus.cc:348] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } }
I20260928 16:38:13.819270 22112 raft_consensus.cc:374] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:38:13.819392 22112 raft_consensus.cc:729] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 164c354707c841caa5fdcbce00c89078, State: Initialized, Role: FOLLOWER
I20260928 16:38:13.819058 22113 raft_consensus.cc:348] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } }
I20260928 16:38:13.819571 22113 raft_consensus.cc:374] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:38:13.819732 22113 raft_consensus.cc:729] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deaf8b17f4054bd9a7e2712b0a28dc64, State: Initialized, Role: FOLLOWER
I20260928 16:38:13.820353 22113 consensus_queue.cc:260] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } }
I20260928 16:38:13.820367 22112 consensus_queue.cc:260] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } }
I20260928 16:38:13.822535 21958 heartbeater.cc:503] Master 127.10.247.254:35653 was elected leader, sending a full tablet report...
I20260928 16:38:13.822971 22113 ts_tablet_manager.cc:1428] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260928 16:38:13.823237 22112 ts_tablet_manager.cc:1428] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
W20260928 16:38:13.905459 22093 tablet.cc:2369] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:38:13.917490 21825 tablet.cc:2369] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:38:13.991293 22119 raft_consensus.cc:482] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:38:13.991775 22119 raft_consensus.cc:504] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } }
I20260928 16:38:13.994222 22119 leader_election.cc:288] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 164c354707c841caa5fdcbce00c89078 (127.10.247.194:46153), d2da7b827019472aaad5a34795567038 (127.10.247.195:38351)
I20260928 16:38:14.002607 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.994756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:44623 (local address 127.10.247.195:38351)
0928 16:38:13.994984 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:13.994991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:13.996179 (+ 1188us) server_negotiation.cc:407] Connection header received
0928 16:38:13.996406 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:13.996410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:13.996488 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:13.996574 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:38:13.997312 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.998148 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.000446 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.001399 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.001593 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:14.001692 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:14.002120 (+ 428us) server_negotiation.cc:299] Negotiation successful
0928 16:38:14.002290 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:38:14.002640 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.994666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:56201 (local address 127.10.247.194:46153)
0928 16:38:13.994967 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:13.994975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:13.996273 (+ 1298us) server_negotiation.cc:407] Connection header received
0928 16:38:13.996473 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:13.996477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:13.996559 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:13.996680 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:38:13.997323 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.998145 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.000444 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.001420 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.001593 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:14.001725 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:14.002108 (+ 383us) server_negotiation.cc:299] Negotiation successful
0928 16:38:14.002280 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260928 16:38:14.002836 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.995268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:46153 (local address 127.10.247.193:56201)
0928 16:38:13.996145 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:13.996166 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:13.996323 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:13.996703 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:13.996707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:13.996732 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:38:13.997161 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.997172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.998291 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.998296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:14.000248 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:14.000266 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.001734 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.001741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:14.001910 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:14.001956 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 16:38:14.002054 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 16:38:14.002281 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":174,"thread_start_us":509,"threads_started":1}
I20260928 16:38:14.002884 22120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:13.995056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:38351 (local address 127.10.247.193:44623)
0928 16:38:13.996039 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0928 16:38:13.996060 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:38:13.996236 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:38:13.996646 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:38:13.996664 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:38:13.996691 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:38:13.997153 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:13.997164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:13.998318 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:13.998324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:14.000256 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:38:14.000271 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.001688 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.001695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:38:14.001857 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:14.001952 (+ 95us) client_negotiation.cc:769] Sending connection context
0928 16:38:14.002051 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 16:38:14.002281 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":158,"thread_start_us":513,"threads_started":1}
I20260928 16:38:14.004174 21913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aabc0ceae8d43119bdb3c443c24494c" candidate_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "164c354707c841caa5fdcbce00c89078" is_pre_election: true
I20260928 16:38:14.004170 22047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aabc0ceae8d43119bdb3c443c24494c" candidate_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2da7b827019472aaad5a34795567038" is_pre_election: true
I20260928 16:38:14.004829 21913 raft_consensus.cc:2454] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate deaf8b17f4054bd9a7e2712b0a28dc64 in term 0.
I20260928 16:38:14.004828 22047 raft_consensus.cc:2454] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate deaf8b17f4054bd9a7e2712b0a28dc64 in term 0.
I20260928 16:38:14.005667 21715 leader_election.cc:302] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 164c354707c841caa5fdcbce00c89078, d2da7b827019472aaad5a34795567038, deaf8b17f4054bd9a7e2712b0a28dc64; no voters:
I20260928 16:38:14.006214 22119 raft_consensus.cc:2781] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:38:14.006399 22119 raft_consensus.cc:482] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:38:14.006527 22119 raft_consensus.cc:3037] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:38:14.009341 22119 raft_consensus.cc:504] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } }
I20260928 16:38:14.010210 22119 leader_election.cc:288] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [CANDIDATE]: Term 1 election: Requested vote from peers 164c354707c841caa5fdcbce00c89078 (127.10.247.194:46153), d2da7b827019472aaad5a34795567038 (127.10.247.195:38351)
I20260928 16:38:14.010952 22047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aabc0ceae8d43119bdb3c443c24494c" candidate_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2da7b827019472aaad5a34795567038"
I20260928 16:38:14.010954 21913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aabc0ceae8d43119bdb3c443c24494c" candidate_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "164c354707c841caa5fdcbce00c89078"
I20260928 16:38:14.011278 21913 raft_consensus.cc:3037] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:38:14.011267 22047 raft_consensus.cc:3037] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:38:14.015316 22047 raft_consensus.cc:2454] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate deaf8b17f4054bd9a7e2712b0a28dc64 in term 1.
I20260928 16:38:14.015316 21913 raft_consensus.cc:2454] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate deaf8b17f4054bd9a7e2712b0a28dc64 in term 1.
I20260928 16:38:14.016155 21715 leader_election.cc:302] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 164c354707c841caa5fdcbce00c89078, d2da7b827019472aaad5a34795567038, deaf8b17f4054bd9a7e2712b0a28dc64; no voters:
I20260928 16:38:14.016589 22119 raft_consensus.cc:2781] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:38:14.017258 22119 raft_consensus.cc:686] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 1 LEADER]: Becoming Leader. State: Replica: deaf8b17f4054bd9a7e2712b0a28dc64, State: Running, Role: LEADER
I20260928 16:38:14.017982 22119 consensus_queue.cc:237] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } }
I20260928 16:38:14.024499 21633 catalog_manager.cc:5495] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 reported cstate change: term changed from 0 to 1, leader changed from <none> to deaf8b17f4054bd9a7e2712b0a28dc64 (127.10.247.193). New cstate: current_term: 1 leader_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "deaf8b17f4054bd9a7e2712b0a28dc64" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 38695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 } health_report { overall_health: UNKNOWN } } }
W20260928 16:38:14.036250 21959 tablet.cc:2369] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:38:14.065261 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:14.059082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.10.247.193:38695)
0928 16:38:14.059376 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:14.059381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:14.059403 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:38:14.059669 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:14.059674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:14.059775 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:14.059927 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:38:14.060284 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.061211 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.062731 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.063166 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.063263 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:14.063836 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:38:14.064561 (+ 725us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:38:14.064764 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:14.064841 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 16:38:14.064936 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260928 16:38:14.075821 22047 raft_consensus.cc:1261] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [term 1 FOLLOWER]: Refusing update from remote peer deaf8b17f4054bd9a7e2712b0a28dc64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:38:14.075816 21913 raft_consensus.cc:1261] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [term 1 FOLLOWER]: Refusing update from remote peer deaf8b17f4054bd9a7e2712b0a28dc64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:38:14.077042 22122 consensus_queue.cc:1035] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2da7b827019472aaad5a34795567038" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 38351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:38:14.077433 22119 consensus_queue.cc:1035] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "164c354707c841caa5fdcbce00c89078" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:38:14.091508 22125 mvcc.cc:205] Tried to move back new op lower bound from 7334352871722229760 to 7334352871503134720. Current Snapshot: MvccSnapshot[applied={T|T < 7334352871722229760}]
I20260928 16:38:14.093891 22127 mvcc.cc:205] Tried to move back new op lower bound from 7334352871722229760 to 7334352871503134720. Current Snapshot: MvccSnapshot[applied={T|T < 7334352871722229760}]
I20260928 16:38:14.110234 21633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:38:14.110596 21633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:38:14.115597 21633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b448bae7e964005b1517e57b3e70615: Sending DeleteTablet for 3 replicas of tablet 2aabc0ceae8d43119bdb3c443c24494c
I20260928 16:38:14.117349 21759 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aabc0ceae8d43119bdb3c443c24494c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:38:14 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:46388
I20260928 16:38:14.117342 21893 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aabc0ceae8d43119bdb3c443c24494c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:38:14 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:33812
I20260928 16:38:14.117343 22027 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aabc0ceae8d43119bdb3c443c24494c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:38:14 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:44316
I20260928 16:38:14.120381 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 21695
I20260928 16:38:14.129380 22134 tablet_replica.cc:330] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: stopping tablet replica
I20260928 16:38:14.130949 22136 tablet_replica.cc:330] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: stopping tablet replica
I20260928 16:38:14.131687 22136 raft_consensus.cc:2229] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:38:14.133622 22134 raft_consensus.cc:2229] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:38:14.134397 22134 raft_consensus.cc:2258] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:38:14.135124 22136 raft_consensus.cc:2258] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:38:14.136137 22135 tablet_replica.cc:330] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: stopping tablet replica
I20260928 16:38:14.136844 22135 raft_consensus.cc:2229] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:38:14.137509 22135 raft_consensus.cc:2258] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:38:14.138258 22136 ts_tablet_manager.cc:1905] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:38:14.140036 22135 ts_tablet_manager.cc:1905] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 16:38:14 dist-test-slave-9fn6 krb5kdc[21589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:38:14.148115 22135 ts_tablet_manager.cc:1918] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:38:14.148360 22135 log.cc:1196] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/2aabc0ceae8d43119bdb3c443c24494c
I20260928 16:38:14.148234 22136 ts_tablet_manager.cc:1918] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:38:14.149583 22134 ts_tablet_manager.cc:1905] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:38:14.148887 22136 log.cc:1196] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/2aabc0ceae8d43119bdb3c443c24494c
I20260928 16:38:14.150568 22135 ts_tablet_manager.cc:1939] T 2aabc0ceae8d43119bdb3c443c24494c P 164c354707c841caa5fdcbce00c89078: Deleting consensus metadata
I20260928 16:38:14.153882 21618 catalog_manager.cc:4841] TS 164c354707c841caa5fdcbce00c89078 (127.10.247.194:46153): tablet 2aabc0ceae8d43119bdb3c443c24494c (table test-table [id=d9c4ae5c9e5249f98ca6b7555cf05bd2]) successfully deleted
I20260928 16:38:14.154119 22136 ts_tablet_manager.cc:1939] T 2aabc0ceae8d43119bdb3c443c24494c P deaf8b17f4054bd9a7e2712b0a28dc64: Deleting consensus metadata
I20260928 16:38:14.156594 21621 catalog_manager.cc:4841] TS deaf8b17f4054bd9a7e2712b0a28dc64 (127.10.247.193:38695): tablet 2aabc0ceae8d43119bdb3c443c24494c (table test-table [id=d9c4ae5c9e5249f98ca6b7555cf05bd2]) successfully deleted
I20260928 16:38:14.158277 22134 ts_tablet_manager.cc:1918] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:38:14.158478 22134 log.cc:1196] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/2aabc0ceae8d43119bdb3c443c24494c
I20260928 16:38:14.158931 22134 ts_tablet_manager.cc:1939] T 2aabc0ceae8d43119bdb3c443c24494c P d2da7b827019472aaad5a34795567038: Deleting consensus metadata
I20260928 16:38:14.161022 21621 catalog_manager.cc:4841] TS d2da7b827019472aaad5a34795567038 (127.10.247.195:38351): tablet 2aabc0ceae8d43119bdb3c443c24494c (table test-table [id=d9c4ae5c9e5249f98ca6b7555cf05bd2]) successfully deleted
I20260928 16:38:14.161216 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:14.122334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.10.247.193:38695)
0928 16:38:14.122616 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:14.122625 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:14.133338 (+ 10713us) server_negotiation.cc:407] Connection header received
0928 16:38:14.133767 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:14.133774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:14.133915 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:14.134007 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:14.138043 (+ 4036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.139336 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.141381 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.142152 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.142264 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:14.148663 (+ 6399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:14.148690 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:14.148716 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:14.148798 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:14.155362 (+ 6564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:14.156779 (+ 1417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:14.156792 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:14.156797 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:14.157203 (+ 406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:14.157785 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:14.157791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:14.157794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:14.158398 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:14.160155 (+ 1757us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:14.160356 (+ 201us) server_negotiation.cc:299] Negotiation successful
0928 16:38:14.160782 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260928 16:38:14.162140 21819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:14.272708 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 21695
I20260928 16:38:14.291328 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 21830
Sep 28 16:38:14 dist-test-slave-9fn6 krb5kdc[21589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:38:14.308712 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:14.292363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.10.247.194:46153)
0928 16:38:14.292548 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:14.292553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:14.292579 (+ 26us) server_negotiation.cc:407] Connection header received
0928 16:38:14.292768 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:14.292775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:14.292863 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:14.292985 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:14.294476 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.295413 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.296923 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.297399 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.297510 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:14.302235 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:14.302256 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:14.302279 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:14.302330 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:14.305567 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:14.306406 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:14.306415 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:14.306418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:14.306585 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:14.306971 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:14.306974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:14.306976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:14.307467 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:14.307782 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:14.308073 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 16:38:14.308315 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:38:14.309567 21953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:14.408512 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 21830
I20260928 16:38:14.426352 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 21964
Sep 28 16:38:14 dist-test-slave-9fn6 krb5kdc[21589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:38:14.443634 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:38:14.427300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44328 (local address 127.10.247.195:38351)
0928 16:38:14.427491 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 16:38:14.427497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:38:14.427518 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:38:14.427680 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:38:14.427685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:38:14.427765 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:38:14.427860 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:38:14.429436 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.430307 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.431721 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:38:14.432176 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:38:14.432293 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:38:14.437267 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:38:14.437293 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:38:14.437315 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:38:14.437375 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:38:14.440360 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:14.441290 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:14.441300 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:14.441302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:14.441503 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:38:14.441857 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:38:14.441861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:38:14.441864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:38:14.442341 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:38:14.442601 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0928 16:38:14.443068 (+ 467us) server_negotiation.cc:299] Negotiation successful
0928 16:38:14.443293 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:38:14.444389 22087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:14.541950 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 21964
I20260928 16:38:14.559046 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 21602
I20260928 16:38:14.560554 21664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:38:14.686048 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 21602
2026-09-28T16:38:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4877 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:38:14 dist-test-slave-9fn6 krb5kdc[22144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:38:14 dist-test-slave-9fn6 krb5kdc[22144](info): set up 2 sockets
Sep 28 16:38:14 dist-test-slave-9fn6 krb5kdc[22144](info): commencing operation
krb5kdc: starting...
W20260928 16:38:15.742596 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:38:15 dist-test-slave-9fn6 krb5kdc[22144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:38:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:38:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.247.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.247.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/rangeradmin_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/rangeradmin_127.10.247.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.247.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.247.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/rangerlookup_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/rangerlookup_127.10.247.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.247.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.247.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/HTTP_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/HTTP_127.10.247.212@KRBTEST.COM.keytab.
I20260928 16:38:16.164848 11231 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKneNwT/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/postgres -l logfile start
2026-09-28 16:38:18.843 UTC [22174] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 16:38:18.843 UTC [22174] LOG: listening on IPv4 address "127.10.247.212", port 51873
2026-09-28 16:38:18.846 UTC [22174] LOG: listening on Unix socket "/tmp/.s.PGSQL.51873"
2026-09-28 16:38:18.870 UTC [22178] LOG: database system was shut down at 2026-09-28 16:38:17 UTC
2026-09-28 16:38:18.874 UTC [22174] LOG: database system is ready to accept connections
I20260928 16:38:19.785141 11231 mini_postgres.cc:96] Postgres bound to 51873
2026-09-28 16:38:19.807 UTC [22186] FATAL: database "slave" does not exist
127.10.247.212:51873 - accepting connections
I20260928 16:38:19.808813 11231 mini_ranger.cc:161] Starting Ranger...
I20260928 16:38:19.842911 11231 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 16:38:20.376780 11231 mini_ranger.cc:87] Created ranger Postgres database
I20260928 16:38:20.376937 11231 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-admin
2026-09-28 16:38:20,701 [I] DB FLAVOR :POSTGRES
2026-09-28 16:38:20,702 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 16:38:20,702 [I] Checking connection..
2026-09-28 16:38:20,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 16:38:20,972 [I] Checking connection passed.
2026-09-28 16:38:20,973 [I] --------- Verifying version history table ---------
2026-09-28 16:38:20,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 16:38:21,256 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 16:38:21,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 16:38:21,539 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 16:38:21,539 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 16:38:21,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 16:38:21.793 UTC [22251] WARNING: there is no transaction in progress
2026-09-28 16:38:21,800 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 16:38:21,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 16:38:22,078 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 16:38:22,078 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 16:38:22,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 16:38:22,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 16:38:22,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9fn6.c.gcp-upstream.internal','N') ;"
2026-09-28 16:38:22,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 16:38:23,100 [I] Table x_portal_user does not exist in database ranger
2026-09-28 16:38:23,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 16:38:23,368 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 16:38:23,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 16:38:23,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 16:38:23,896 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 16:38:23,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 16:38:24.949 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:24.966 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:24.984 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.001 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.019 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.039 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.099 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.107 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.124 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.136 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.148 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.162 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.173 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.181 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.189 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25.197 UTC [22360] WARNING: there is no transaction in progress
2026-09-28 16:38:25,875 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 16:38:25,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 16:38:26,135 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 16:38:26,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 16:38:26,396 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 16:38:26,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 16:38:26,660 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 16:38:26,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 16:38:26,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 16:38:27,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9fn6.c.gcp-upstream.internal';"
2026-09-28 16:38:27,408 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 16:38:27,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 16:38:27,678 [I] DB_PATCHES have already been applied
I20260928 16:38:27.685482 11231 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKneNwT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 16:38:27.685591 11231 mini_ranger.cc:193] Using host: 127.10.247.212
I20260928 16:38:27.705346 11231 mini_ranger.cc:227] Ranger admin URL: http://127.10.247.212:59901
Sep 28, 2026 4:38:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 4:38:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 4:38:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 4:38:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 4:38:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 4:38:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 4:38:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 16:38:28.761399 11231 mini_ranger.cc:160] Time spent starting Ranger: real 8.953s user 0.006s sys 0.055s
Sep 28, 2026 4:38:50 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 4:38:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 28, 2026 4:38:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 4:38:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 4:38:51 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 16:38:52.580303 11231 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.247.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.247.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/rangerkms_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/rangerkms_127.10.247.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.247.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.247.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.247.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/HTTP_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.247.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/HTTP_127.10.247.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 16:38:52 dist-test-slave-9fn6 krb5kdc[22144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613532, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 16:38:52.778807 11231 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 16:38:52.806212 11231 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 16:38:53.711089 11231 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 16:38:53.711259 11231 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-kms
I20260928 16:38:53.711426 11231 mini_ranger_kms.cc:227] Using postgres at 127.10.247.212:51873
2026-09-28 16:38:54,675 [I] DB FLAVOR :POSTGRES
2026-09-28 16:38:54,675 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 16:38:54,675 [I] Checking connection
2026-09-28 16:38:54,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 16:38:54,942 [I] connection success
2026-09-28 16:38:54,942 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 16:38:54,942 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 16:38:54,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 16:38:55,188 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 16:38:55,189 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 16:38:55,189 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 16:38:55,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:51873/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 16:38:55,466 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 16:38:56.008926 11231 mini_ranger_kms.cc:310] Created kms service
I20260928 16:38:56.246457 11231 mini_ranger_kms.cc:326] Created kudu user
I20260928 16:38:56.295212 11231 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 16:38:56 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613532, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:38:56.744762 11231 mini_ranger_kms.cc:384] Added ranger policy
I20260928 16:38:56.746321 11231 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 16:38:56.746417 11231 mini_ranger_kms.cc:240] Using host: 127.10.247.212
I20260928 16:38:56.776803 11231 mini_ranger_kms.cc:276] Ranger KMS PID: 22542
I20260928 16:38:56.777073 11231 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.247.212:42647
Sep 28, 2026 4:38:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 4:38:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 4:38:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 4:38:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 4:38:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 4:38:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 16:38:58.943783 11231 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.165s user 0.166s sys 0.467s
I20260928 16:38:58.944062 11231 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 16:38:58.944310 11231 mini_ranger_kms.cc:401] 127.10.247.212:42647/kms/v1/keys
Sep 28 16:38:59 dist-test-slave-9fn6 krb5kdc[22144](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790613539, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.247.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 16:38:59 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613539, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.247.212@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
Sep 28 16:39:00 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613539, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.247.212@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
Sep 28 16:39:02 dist-test-slave-9fn6 krb5kdc[22144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:39:02.796908 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:33609
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:33609
--ranger_config_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:42647/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:39:03.319597 22593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:39:03.320302 22593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:39:03.320564 22593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:39:03.330616 22593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:39:03.330794 22593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:39:03.330893 22593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:39:03.330991 22593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:39:03.331067 22593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:39:03.343091 22593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.247.212:42647/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:33609
--ranger_config_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:33609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:39:03.345058 22593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:39:03.346796 22593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:39:03.361109 22598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:39:03.361141 22601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:39:03.361090 22599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:39:03.363464 22593 server_base.cc:1034] running on GCE node
I20260928 16:39:03.364782 22593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:39:03.367111 22593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:39:03.368372 22593 hybrid_clock.cc:648] HybridClock initialized: now 1790613543368314 us; error 40 us; skew 500 ppm
I20260928 16:39:03.368888 22593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:39:03 dist-test-slave-9fn6 krb5kdc[22144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613543, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:39:03.373658 22593 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:39:03.375592 22593 webserver.cc:460] Webserver started at http://127.10.247.254:37107/ using document root <none> and password file <none>
I20260928 16:39:03.376405 22593 fs_manager.cc:359] Metadata directory not provided
I20260928 16:39:03.376508 22593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:39:03.376914 22593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 16:39:03 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613543, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:39:03.450198 22593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "5f35399965354bff93de65c195711db3"
format_stamp: "Formatted at 2026-09-28 16:39:03 on dist-test-slave-9fn6"
server_key: "f4dfa49ecbaee26cc2b76f7ed710e5ff"
server_key_iv: "4330d1e6c9b97cf7d2a3783097ed940f"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:03.451280 22593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "5f35399965354bff93de65c195711db3"
format_stamp: "Formatted at 2026-09-28 16:39:03 on dist-test-slave-9fn6"
server_key: "f4dfa49ecbaee26cc2b76f7ed710e5ff"
server_key_iv: "4330d1e6c9b97cf7d2a3783097ed940f"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:03.458258 22593 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260928 16:39:03.485965 22608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:39:03.488165 22593 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260928 16:39:03.488481 22593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "5f35399965354bff93de65c195711db3"
format_stamp: "Formatted at 2026-09-28 16:39:03 on dist-test-slave-9fn6"
server_key: "f4dfa49ecbaee26cc2b76f7ed710e5ff"
server_key_iv: "4330d1e6c9b97cf7d2a3783097ed940f"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:03.488914 22593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:39:03.522306 22593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:39:03.527709 22593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:39:03.528100 22593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:39:03.560046 22593 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:33609
I20260928 16:39:03.560070 22659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:33609 every 8 connection(s)
I20260928 16:39:03.561877 22593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:39:03.566558 22660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:39:03.567703 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 22593
I20260928 16:39:03.567904 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:39:03.568687 11231 external_mini_cluster.cc:1428] Setting key def58eb4e184c846e89d4554fd3acfd5
I20260928 16:39:03.584982 22660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3: Bootstrap starting.
Sep 28 16:39:03 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:39:03.589963 22660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3: Neither blocks nor log segments found. Creating new log.
I20260928 16:39:03.592690 22660 log.cc:824] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3: Log is configured to *not* fsync() on all Append() calls
I20260928 16:39:03.597014 22660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3: No bootstrap required, opened a new log
I20260928 16:39:03.597348 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:03.571263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.10.247.254:33609)
0928 16:39:03.572562 (+ 1299us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:03.572580 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:03.572675 (+ 95us) server_negotiation.cc:407] Connection header received
0928 16:39:03.574940 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:03.574976 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:03.575715 (+ 739us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:03.576303 (+ 588us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:39:03.577865 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:03.580014 (+ 2149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:03.581477 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:03.582728 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:03.582915 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:03.587402 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:39:03.587416 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:39:03.587438 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:39:03.587515 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:39:03.592118 (+ 4603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:03.592954 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:03.592963 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:03.592972 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:03.593185 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:03.593732 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:03.593735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:03.593738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:03.594449 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:39:03.594827 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:03.595253 (+ 426us) server_negotiation.cc:299] Negotiation successful
0928 16:39:03.595730 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1014,"thread_start_us":701,"threads_started":1}
I20260928 16:39:03.606757 22660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f35399965354bff93de65c195711db3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33609 } }
I20260928 16:39:03.607371 22660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:39:03.607553 22660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f35399965354bff93de65c195711db3, State: Initialized, Role: FOLLOWER
I20260928 16:39:03.608392 22660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f35399965354bff93de65c195711db3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33609 } }
I20260928 16:39:03.608855 22660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:39:03.609110 22660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:39:03.609392 22660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:39:03.612661 22660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f35399965354bff93de65c195711db3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33609 } }
I20260928 16:39:03.613297 22660 leader_election.cc:302] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f35399965354bff93de65c195711db3; no voters:
I20260928 16:39:03.614380 22660 leader_election.cc:288] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:39:03.614639 22665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:39:03.615857 22665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [term 1 LEADER]: Becoming Leader. State: Replica: 5f35399965354bff93de65c195711db3, State: Running, Role: LEADER
I20260928 16:39:03.616600 22665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f35399965354bff93de65c195711db3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33609 } }
I20260928 16:39:03.617170 22660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:39:03.620428 22667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f35399965354bff93de65c195711db3. Latest consensus state: current_term: 1 leader_uuid: "5f35399965354bff93de65c195711db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f35399965354bff93de65c195711db3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33609 } } }
I20260928 16:39:03.620447 22666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f35399965354bff93de65c195711db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f35399965354bff93de65c195711db3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33609 } } }
I20260928 16:39:03.621014 22667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [sys.catalog]: This master's current role is: LEADER
I20260928 16:39:03.621025 22666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3 [sys.catalog]: This master's current role is: LEADER
I20260928 16:39:03.629594 22671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:39:03.638468 22671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:39:03.639464 22660 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 16:39:03.674396 22671 catalog_manager.cc:1347] Generated new cluster ID: 443dd9d0347448feabb49ce865b28e39
I20260928 16:39:03.674597 22671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:39:03.705777 22671 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:39:03.708572 22671 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:39:03.721438 22671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3: Generated new TSK 0
I20260928 16:39:03.723759 22671 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:39:04.551043 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:42647/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33609
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:39:04 dist-test-slave-9fn6 krb5kdc[22144](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790613544, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 16:39:05.109776 22706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:39:05.110306 22706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:39:05.110543 22706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:39:05.123401 22706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:39:05.123594 22706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:39:05.123847 22706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:39:05.140933 22706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.247.212:42647/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:39:05.143074 22706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:39:05.145534 22706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:39:05.172525 22717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:39:05.174449 22716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:39:05.176149 22720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:39:05.177013 22706 server_base.cc:1034] running on GCE node
I20260928 16:39:05.177913 22706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:39:05.179286 22706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:39:05.186864 22706 hybrid_clock.cc:648] HybridClock initialized: now 1790613545185139 us; error 1659 us; skew 500 ppm
I20260928 16:39:05.187475 22706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:39:05 dist-test-slave-9fn6 krb5kdc[22144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613545, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:39:05.199036 22706 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:39:05.202302 22706 webserver.cc:460] Webserver started at http://127.10.247.193:34173/ using document root <none> and password file <none>
I20260928 16:39:05.203667 22706 fs_manager.cc:359] Metadata directory not provided
I20260928 16:39:05.203953 22706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:39:05.204555 22706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 16:39:05 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613545, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:39:05.238780 22706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "94f9df6bbe714a03ae43814617823f2a"
format_stamp: "Formatted at 2026-09-28 16:39:05 on dist-test-slave-9fn6"
server_key: "c87cf17eba379e7380fd28bc57d9eb2f"
server_key_iv: "8188aaf94d27860633d77f677dc6bdce"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:05.240648 22706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "94f9df6bbe714a03ae43814617823f2a"
format_stamp: "Formatted at 2026-09-28 16:39:05 on dist-test-slave-9fn6"
server_key: "c87cf17eba379e7380fd28bc57d9eb2f"
server_key_iv: "8188aaf94d27860633d77f677dc6bdce"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:05.250461 22706 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.003s
I20260928 16:39:05.285949 22726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:39:05.288750 22706 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.011s sys 0.001s
I20260928 16:39:05.289136 22706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "94f9df6bbe714a03ae43814617823f2a"
format_stamp: "Formatted at 2026-09-28 16:39:05 on dist-test-slave-9fn6"
server_key: "c87cf17eba379e7380fd28bc57d9eb2f"
server_key_iv: "8188aaf94d27860633d77f677dc6bdce"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:05.289568 22706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:39:05.343986 22706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:39:05.350359 22706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:39:05.350737 22706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:39:05.352090 22706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:39:05.354035 22706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:39:05.354131 22706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:39:05.354295 22706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:39:05.354354 22706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:39:05 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613544, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:39:05.478683 22706 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:45751
I20260928 16:39:05.479038 22839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:45751 every 8 connection(s)
I20260928 16:39:05.480829 22706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:39:05.482231 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 22706
I20260928 16:39:05.482462 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:39:05.483256 11231 external_mini_cluster.cc:1428] Setting key e256db54901db459aad702967df3c105
Sep 28 16:39:05 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613545, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:39:05.519726 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:05.489615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:34357 (local address 127.10.247.254:33609)
0928 16:39:05.490215 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:05.490222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:05.491554 (+ 1332us) server_negotiation.cc:407] Connection header received
0928 16:39:05.495840 (+ 4286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:05.495848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:05.495962 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:05.496237 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:39:05.500141 (+ 3904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:05.501628 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:05.504859 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:05.505615 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:05.505733 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:05.511069 (+ 5336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:39:05.511077 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:39:05.511082 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:39:05.511143 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:39:05.515021 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:05.516290 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:05.516297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:05.516300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:05.516499 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:05.517204 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:05.517209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:05.517212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:05.517641 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:39:05.517782 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:05.519023 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0928 16:39:05.519318 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":280,"threads_started":1}
I20260928 16:39:05.523695 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:05.490443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:33609 (local address 127.10.247.193:34357)
0928 16:39:05.491363 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0928 16:39:05.491419 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:39:05.495306 (+ 3887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:39:05.497209 (+ 1903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:39:05.497225 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:39:05.498363 (+ 1138us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:39:05.499878 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:05.499913 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:05.501988 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:05.501994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:05.504571 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:05.504597 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:05.506197 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:05.506204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:05.506559 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:05.507539 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:39:05.507581 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:39:05.510730 (+ 3149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:39:05.515343 (+ 4613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:39:05.515355 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:39:05.515382 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:39:05.516122 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:39:05.516715 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:39:05.516720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:39:05.516723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:39:05.517068 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:39:05.517900 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:39:05.517909 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:39:05.518498 (+ 589us) client_negotiation.cc:769] Sending connection context
0928 16:39:05.521145 (+ 2647us) client_negotiation.cc:240] Negotiation successful
0928 16:39:05.521886 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":7548,"threads_started":1}
I20260928 16:39:05.528230 22840 heartbeater.cc:347] Connected to a master server at 127.10.247.254:33609
I20260928 16:39:05.528741 22840 heartbeater.cc:464] Registering TS with master...
I20260928 16:39:05.529946 22840 heartbeater.cc:511] Master 127.10.247.254:33609 requested a full tablet report, sending...
I20260928 16:39:05.533756 22625 ts_manager.cc:194] Registered new tserver with Master: 94f9df6bbe714a03ae43814617823f2a (127.10.247.193:45751)
I20260928 16:39:05.536752 22625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:34357
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
W20260928 16:39:05.603924 11231 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.247.194: real 0.117s user 0.000s sys 0.026s
I20260928 16:39:05.604863 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:42647/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33609
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:39:05 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613544, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
W20260928 16:39:06.261049 22846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:39:06.261977 22846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:39:06.262444 22846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:39:06.276732 22846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:39:06.277148 22846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:39:06.277575 22846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:39:06.300132 22846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.247.212:42647/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:39:06.302691 22846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:39:06.305263 22846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:39:06.330792 22860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:39:06.335641 22861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:39:06.339423 22863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:39:06.342525 22846 server_base.cc:1034] running on GCE node
I20260928 16:39:06.343528 22846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:39:06.345199 22846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:39:06.346565 22846 hybrid_clock.cc:648] HybridClock initialized: now 1790613546346484 us; error 68 us; skew 500 ppm
I20260928 16:39:06.347023 22846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:39:06 dist-test-slave-9fn6 krb5kdc[22144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613546, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:39:06.352649 22846 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:39:06.355862 22846 webserver.cc:460] Webserver started at http://127.10.247.194:39259/ using document root <none> and password file <none>
I20260928 16:39:06.357125 22846 fs_manager.cc:359] Metadata directory not provided
I20260928 16:39:06.357295 22846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:39:06.357733 22846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 16:39:06 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613546, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:39:06.383139 22846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "5777675a7ace4d72b8afef677e931ac2"
format_stamp: "Formatted at 2026-09-28 16:39:06 on dist-test-slave-9fn6"
server_key: "c8316a8cf2c623d756e48699cfefa8ce"
server_key_iv: "8d4a788d1a096f615ce76e707c680fe6"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:06.385727 22846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "5777675a7ace4d72b8afef677e931ac2"
format_stamp: "Formatted at 2026-09-28 16:39:06 on dist-test-slave-9fn6"
server_key: "c8316a8cf2c623d756e48699cfefa8ce"
server_key_iv: "8d4a788d1a096f615ce76e707c680fe6"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:06.416318 22846 fs_manager.cc:691] Time spent creating directory manager: real 0.030s user 0.012s sys 0.011s
I20260928 16:39:06.458501 22870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:39:06.462775 22846 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.004s sys 0.008s
I20260928 16:39:06.463227 22846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "5777675a7ace4d72b8afef677e931ac2"
format_stamp: "Formatted at 2026-09-28 16:39:06 on dist-test-slave-9fn6"
server_key: "c8316a8cf2c623d756e48699cfefa8ce"
server_key_iv: "8d4a788d1a096f615ce76e707c680fe6"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:06.463835 22846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:39:06.541905 22840 heartbeater.cc:503] Master 127.10.247.254:33609 was elected leader, sending a full tablet report...
I20260928 16:39:06.559311 22846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:39:06.576294 22846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:39:06.577268 22846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:39:06.587111 22846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:39:06.598075 22846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:39:06.598349 22846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:39:06.598599 22846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:39:06.598704 22846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:39:06.708592 22846 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:34097
I20260928 16:39:06.708685 22982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:34097 every 8 connection(s)
I20260928 16:39:06.711424 22846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:39:06.712672 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 22846
I20260928 16:39:06.712882 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:39:06.713546 11231 external_mini_cluster.cc:1428] Setting key e21b40a6d8ec09fd7cceacb3e5c582e4
Sep 28 16:39:06 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613546, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
I20260928 16:39:06.775701 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:06.719464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:33609 (local address 127.10.247.194:51247)
0928 16:39:06.740849 (+ 21385us) negotiation.cc:107] Waiting for socket to connect
0928 16:39:06.740940 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0928 16:39:06.744146 (+ 3206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:39:06.747535 (+ 3389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:39:06.747557 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:39:06.748566 (+ 1009us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:39:06.750003 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:06.750049 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:06.754476 (+ 4427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:06.754485 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:06.757279 (+ 2794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:06.757314 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:06.758811 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:06.758821 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:06.759253 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:06.760288 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:39:06.760326 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:39:06.763626 (+ 3300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:39:06.768489 (+ 4863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:39:06.768504 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:39:06.768534 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:39:06.769543 (+ 1009us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:39:06.770271 (+ 728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:39:06.770277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:39:06.770282 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:39:06.770873 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:39:06.771808 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:39:06.771817 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:39:06.772497 (+ 680us) client_negotiation.cc:769] Sending connection context
0928 16:39:06.772964 (+ 467us) client_negotiation.cc:240] Negotiation successful
0928 16:39:06.773786 (+ 822us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20840,"thread_start_us":20478,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
I20260928 16:39:06.777817 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:06.725006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:51247 (local address 127.10.247.254:33609)
0928 16:39:06.746563 (+ 21557us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:06.746571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:06.746607 (+ 36us) server_negotiation.cc:407] Connection header received
0928 16:39:06.746699 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:06.746705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:06.746817 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:06.746998 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:39:06.750393 (+ 3395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:06.754139 (+ 3746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:06.757662 (+ 3523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:06.758408 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:06.759896 (+ 1488us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:06.764058 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:39:06.764066 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:39:06.764072 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:39:06.764136 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:39:06.768186 (+ 4050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:06.769801 (+ 1615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:06.769807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:06.769811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:06.770106 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:06.771105 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:06.771110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:06.771114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:06.771632 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:39:06.776758 (+ 5126us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:06.777016 (+ 258us) server_negotiation.cc:299] Negotiation successful
0928 16:39:06.777347 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21342,"thread_start_us":22122,"threads_started":1}
I20260928 16:39:06.779747 22983 heartbeater.cc:347] Connected to a master server at 127.10.247.254:33609
I20260928 16:39:06.780198 22983 heartbeater.cc:464] Registering TS with master...
I20260928 16:39:06.781435 22983 heartbeater.cc:511] Master 127.10.247.254:33609 requested a full tablet report, sending...
I20260928 16:39:06.784011 22625 ts_manager.cc:194] Registered new tserver with Master: 5777675a7ace4d72b8afef677e931ac2 (127.10.247.194:34097)
I20260928 16:39:06.785588 22625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:51247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
W20260928 16:39:06.828052 11231 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.247.195: real 0.110s user 0.000s sys 0.029s
I20260928 16:39:06.829479 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:42647/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33609
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:39:07.326918 22989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:39:07.327342 22989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:39:07.327556 22989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:39:07.335969 22989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:39:07.336148 22989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:39:07.336345 22989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:39:07.348549 22989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.247.212:42647/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:39:07.350495 22989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:39:07.352303 22989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:39:07.368083 22995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:39:07.368698 22997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:39:07.368741 22994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:39:07.369165 22989 server_base.cc:1034] running on GCE node
I20260928 16:39:07.369967 22989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:39:07.371263 22989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:39:07.372504 22989 hybrid_clock.cc:648] HybridClock initialized: now 1790613547372431 us; error 73 us; skew 500 ppm
I20260928 16:39:07.372982 22989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:39:07 dist-test-slave-9fn6 krb5kdc[22144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613547, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:39:07.376924 22989 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:39:07.378784 22989 webserver.cc:460] Webserver started at http://127.10.247.195:44999/ using document root <none> and password file <none>
I20260928 16:39:07.379693 22989 fs_manager.cc:359] Metadata directory not provided
I20260928 16:39:07.379827 22989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:39:07.380189 22989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 16:39:07 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613547, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:39:07.402081 22989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "d6f92fdb2c9148e5bd37279c64c51e08"
format_stamp: "Formatted at 2026-09-28 16:39:07 on dist-test-slave-9fn6"
server_key: "3f94b471feea6eeb457bfaf2dd55105e"
server_key_iv: "31a689dafcb38faf2fd2c272975ba789"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:07.403481 22989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "d6f92fdb2c9148e5bd37279c64c51e08"
format_stamp: "Formatted at 2026-09-28 16:39:07 on dist-test-slave-9fn6"
server_key: "3f94b471feea6eeb457bfaf2dd55105e"
server_key_iv: "31a689dafcb38faf2fd2c272975ba789"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:07.411170 22989 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260928 16:39:07.429538 23004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:39:07.431365 22989 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260928 16:39:07.431577 22989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "d6f92fdb2c9148e5bd37279c64c51e08"
format_stamp: "Formatted at 2026-09-28 16:39:07 on dist-test-slave-9fn6"
server_key: "3f94b471feea6eeb457bfaf2dd55105e"
server_key_iv: "31a689dafcb38faf2fd2c272975ba789"
server_key_version: "kuduclusterkey@0"
I20260928 16:39:07.431866 22989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:39:07.449687 22989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:39:07.456481 22989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:39:07.457124 22989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:39:07.458926 22989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:39:07.461426 22989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:39:07.461596 22989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:39:07.461833 22989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:39:07.461961 22989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:39:07.526948 22989 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:34135
I20260928 16:39:07.526957 23116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:34135 every 8 connection(s)
I20260928 16:39:07.528808 22989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:39:07.536077 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 22989
I20260928 16:39:07.536281 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:39:07.536871 11231 external_mini_cluster.cc:1428] Setting key 15be9e5bd4c044c16f51d0d8f77f3a74
Sep 28 16:39:07 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613547, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:39:07.552064 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.532033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:45517 (local address 127.10.247.254:33609)
0928 16:39:07.532674 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:07.532682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:07.533700 (+ 1018us) server_negotiation.cc:407] Connection header received
0928 16:39:07.535809 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:07.535815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:07.535922 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:07.536035 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:39:07.537921 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.538874 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.540770 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.541273 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.541387 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.545309 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:39:07.545314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:39:07.545318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:39:07.545357 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:39:07.548508 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:07.549369 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:07.549375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:07.549379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:07.549533 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:07.550005 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:07.550010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:07.550013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:07.550346 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:39:07.550501 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:07.551377 (+ 876us) server_negotiation.cc:299] Negotiation successful
0928 16:39:07.551713 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":287,"threads_started":1}
I20260928 16:39:07.553630 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.532689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:33609 (local address 127.10.247.195:45517)
0928 16:39:07.533511 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0928 16:39:07.533563 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 16:39:07.535505 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:39:07.536330 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:39:07.536345 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:39:07.536928 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:39:07.537743 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.537764 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.539031 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.539035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.540592 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.540637 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.541526 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.541535 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.541823 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.542558 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:39:07.542583 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:39:07.545043 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:39:07.548674 (+ 3631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:39:07.548685 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:39:07.548706 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:39:07.549243 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:39:07.549643 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:39:07.549649 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:39:07.549652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:39:07.549889 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:39:07.550488 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:39:07.550498 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:39:07.550935 (+ 437us) client_negotiation.cc:769] Sending connection context
0928 16:39:07.551269 (+ 334us) client_negotiation.cc:240] Negotiation successful
0928 16:39:07.551943 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":315,"threads_started":1}
I20260928 16:39:07.555366 23117 heartbeater.cc:347] Connected to a master server at 127.10.247.254:33609
I20260928 16:39:07.555766 23117 heartbeater.cc:464] Registering TS with master...
I20260928 16:39:07.556835 23117 heartbeater.cc:511] Master 127.10.247.254:33609 requested a full tablet report, sending...
I20260928 16:39:07.558631 22625 ts_manager.cc:194] Registered new tserver with Master: d6f92fdb2c9148e5bd37279c64c51e08 (127.10.247.195:34135)
I20260928 16:39:07.559661 22625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:45517
I20260928 16:39:07.566213 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:39:07.587152 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.572673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55000 (local address 127.10.247.254:33609)
0928 16:39:07.572864 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:07.572870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:07.573463 (+ 593us) server_negotiation.cc:407] Connection header received
0928 16:39:07.573661 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:07.573666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:07.573757 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:07.573843 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:39:07.575532 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.576363 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.577958 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.578456 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.578584 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.581447 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:39:07.581452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:39:07.581456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:39:07.581498 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:39:07.584149 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:07.584972 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:07.584977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:07.584979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:07.585158 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:07.585565 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:07.585568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:07.585570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:07.585895 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:39:07.586166 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:07.586519 (+ 353us) server_negotiation.cc:299] Negotiation successful
0928 16:39:07.586780 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:39:07.591917 22625 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:39:07.600307 22625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:39:07.628331 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.615907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32900 (local address 127.10.247.194:34097)
0928 16:39:07.618337 (+ 2430us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:07.618345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:07.618371 (+ 26us) server_negotiation.cc:407] Connection header received
0928 16:39:07.618468 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:07.618475 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:07.619820 (+ 1345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:07.620050 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:39:07.621235 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.622470 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.625265 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.626830 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.627026 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.627270 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:07.627694 (+ 424us) server_negotiation.cc:299] Negotiation successful
0928 16:39:07.627906 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2268,"thread_start_us":2138,"threads_started":1}
I20260928 16:39:07.628813 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.615935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59088 (local address 127.10.247.195:34135)
0928 16:39:07.618005 (+ 2070us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:07.618015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:07.618058 (+ 43us) server_negotiation.cc:407] Connection header received
0928 16:39:07.618182 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:07.618191 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:07.618491 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:07.619748 (+ 1257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:39:07.620134 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.621931 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.625136 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.626278 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.626476 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.626714 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:07.627864 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0928 16:39:07.628054 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1870,"thread_start_us":1710,"threads_started":1}
I20260928 16:39:07.627765 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.615497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:34135 (local address 127.0.0.1:59088)
0928 16:39:07.616292 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0928 16:39:07.616343 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 16:39:07.616532 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:39:07.618775 (+ 2243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:39:07.618784 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:39:07.618830 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:39:07.619466 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.619485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.622082 (+ 2597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.622089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.624912 (+ 2823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.624933 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.626522 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.626528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.626807 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.626856 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 16:39:07.627012 (+ 156us) client_negotiation.cc:240] Negotiation successful
0928 16:39:07.627254 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":74,"thread_start_us":600,"threads_started":1}
I20260928 16:39:07.629905 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.615583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34097 (local address 127.0.0.1:32900)
0928 16:39:07.617185 (+ 1602us) negotiation.cc:107] Waiting for socket to connect
0928 16:39:07.617203 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:39:07.617328 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:39:07.620346 (+ 3018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:39:07.620352 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:39:07.620378 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:39:07.621009 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.621026 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.622626 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.622631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.625085 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.625101 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.627197 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.627204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.627464 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.627501 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 16:39:07.627590 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 16:39:07.627736 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1467,"mutex_wait_us":19,"thread_start_us":1304,"threads_started":1}
I20260928 16:39:07.631134 23135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.619072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49368 (local address 127.10.247.193:45751)
0928 16:39:07.619866 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:07.619875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:07.619902 (+ 27us) server_negotiation.cc:407] Connection header received
0928 16:39:07.620025 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:07.620033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:07.620275 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:07.621293 (+ 1018us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:39:07.621606 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.623029 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.627114 (+ 4085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.629421 (+ 2307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.629754 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.629947 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:07.630549 (+ 602us) server_negotiation.cc:299] Negotiation successful
0928 16:39:07.630736 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":430,"threads_started":1}
I20260928 16:39:07.631383 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.618383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:45751 (local address 127.0.0.1:49368)
0928 16:39:07.619071 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0928 16:39:07.619088 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:39:07.619229 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:39:07.620493 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:39:07.620501 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:39:07.620524 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:39:07.621069 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.621079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.623189 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.623194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.626880 (+ 3686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.626894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.630043 (+ 3149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.630051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.630354 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.630382 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 16:39:07.630465 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 16:39:07.630627 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":29,"thread_start_us":419,"threads_started":1}
I20260928 16:39:07.634903 22918 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa5afa5368f45da87f539a6c3d39f7b (DEFAULT_TABLE table=test-table [id=91983168c17c410d9b31383f5e923583]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:39:07.635131 23052 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa5afa5368f45da87f539a6c3d39f7b (DEFAULT_TABLE table=test-table [id=91983168c17c410d9b31383f5e923583]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:39:07.636552 23052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa5afa5368f45da87f539a6c3d39f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:39:07.636560 22918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa5afa5368f45da87f539a6c3d39f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:39:07.643683 22774 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa5afa5368f45da87f539a6c3d39f7b (DEFAULT_TABLE table=test-table [id=91983168c17c410d9b31383f5e923583]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:39:07.645666 22774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa5afa5368f45da87f539a6c3d39f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:39:07.651000 23136 tablet_bootstrap.cc:492] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: Bootstrap starting.
I20260928 16:39:07.655084 23136 tablet_bootstrap.cc:654] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: Neither blocks nor log segments found. Creating new log.
I20260928 16:39:07.656246 23137 tablet_bootstrap.cc:492] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: Bootstrap starting.
I20260928 16:39:07.656843 23136 log.cc:824] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: Log is configured to *not* fsync() on all Append() calls
I20260928 16:39:07.661003 23137 tablet_bootstrap.cc:654] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: Neither blocks nor log segments found. Creating new log.
I20260928 16:39:07.662477 23136 tablet_bootstrap.cc:492] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: No bootstrap required, opened a new log
I20260928 16:39:07.662624 23137 log.cc:824] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: Log is configured to *not* fsync() on all Append() calls
I20260928 16:39:07.662737 23139 tablet_bootstrap.cc:492] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: Bootstrap starting.
I20260928 16:39:07.663059 23136 ts_tablet_manager.cc:1397] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260928 16:39:07.667656 23139 tablet_bootstrap.cc:654] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: Neither blocks nor log segments found. Creating new log.
I20260928 16:39:07.668205 23137 tablet_bootstrap.cc:492] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: No bootstrap required, opened a new log
I20260928 16:39:07.668563 23137 ts_tablet_manager.cc:1397] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.002s
I20260928 16:39:07.670683 23139 log.cc:824] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: Log is configured to *not* fsync() on all Append() calls
I20260928 16:39:07.676487 23136 raft_consensus.cc:348] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } }
I20260928 16:39:07.677364 23136 raft_consensus.cc:374] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:39:07.677529 23136 raft_consensus.cc:729] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6f92fdb2c9148e5bd37279c64c51e08, State: Initialized, Role: FOLLOWER
I20260928 16:39:07.678395 23136 consensus_queue.cc:260] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } }
I20260928 16:39:07.680146 23117 heartbeater.cc:503] Master 127.10.247.254:33609 was elected leader, sending a full tablet report...
I20260928 16:39:07.680986 23136 ts_tablet_manager.cc:1428] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
I20260928 16:39:07.681061 23139 tablet_bootstrap.cc:492] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: No bootstrap required, opened a new log
I20260928 16:39:07.681478 23139 ts_tablet_manager.cc:1397] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.014s
I20260928 16:39:07.681407 23137 raft_consensus.cc:348] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } }
I20260928 16:39:07.681964 23137 raft_consensus.cc:374] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:39:07.682085 23137 raft_consensus.cc:729] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5777675a7ace4d72b8afef677e931ac2, State: Initialized, Role: FOLLOWER
I20260928 16:39:07.682953 23137 consensus_queue.cc:260] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } }
I20260928 16:39:07.685689 22983 heartbeater.cc:503] Master 127.10.247.254:33609 was elected leader, sending a full tablet report...
I20260928 16:39:07.686569 23137 ts_tablet_manager.cc:1428] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: Time spent starting tablet: real 0.018s user 0.011s sys 0.004s
I20260928 16:39:07.692194 23139 raft_consensus.cc:348] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } }
I20260928 16:39:07.692785 23139 raft_consensus.cc:374] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:39:07.692945 23139 raft_consensus.cc:729] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f9df6bbe714a03ae43814617823f2a, State: Initialized, Role: FOLLOWER
I20260928 16:39:07.693727 23139 consensus_queue.cc:260] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } }
I20260928 16:39:07.696146 23139 ts_tablet_manager.cc:1428] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
W20260928 16:39:07.740825 22841 tablet.cc:2369] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:39:07.743403 22984 tablet.cc:2369] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:39:07.780959 23118 tablet.cc:2369] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:39:07.931563 23142 raft_consensus.cc:482] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:39:07.931996 23142 raft_consensus.cc:504] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } }
I20260928 16:39:07.934854 23142 leader_election.cc:288] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5777675a7ace4d72b8afef677e931ac2 (127.10.247.194:34097), 94f9df6bbe714a03ae43814617823f2a (127.10.247.193:45751)
I20260928 16:39:07.942865 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.935423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:53195 (local address 127.10.247.194:34097)
0928 16:39:07.935692 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:07.935700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:07.936093 (+ 393us) server_negotiation.cc:407] Connection header received
0928 16:39:07.936351 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:07.936355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:07.936434 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:07.936540 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:39:07.937011 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.938074 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.940688 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.941901 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.942067 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.942159 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:07.942435 (+ 276us) server_negotiation.cc:299] Negotiation successful
0928 16:39:07.942558 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 16:39:07.942878 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.935423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34097 (local address 127.10.247.195:53195)
0928 16:39:07.935983 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0928 16:39:07.936009 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 16:39:07.936188 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:39:07.936581 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:39:07.936585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:39:07.936640 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:39:07.936871 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.936881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.938243 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.938247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.940490 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.940507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.942068 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.942072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.942247 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.942287 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 16:39:07.942402 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 16:39:07.942562 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":17}
I20260928 16:39:07.943122 23135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.935803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:39761 (local address 127.10.247.193:45751)
0928 16:39:07.936016 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:07.936021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:07.936303 (+ 282us) server_negotiation.cc:407] Connection header received
0928 16:39:07.936591 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:07.936597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:07.936720 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:07.936812 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:39:07.937485 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.938489 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.940922 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.942190 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.942323 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.942384 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:07.942740 (+ 356us) server_negotiation.cc:299] Negotiation successful
0928 16:39:07.942881 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:39:07.943213 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:07.935603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:45751 (local address 127.10.247.195:39761)
0928 16:39:07.936210 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0928 16:39:07.936225 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 16:39:07.936408 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:39:07.936963 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:39:07.936966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:39:07.936987 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:39:07.937315 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.937323 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.938689 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.938694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.940785 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:39:07.940797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:39:07.942396 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:39:07.942401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:39:07.942570 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:07.942611 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 16:39:07.942696 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 16:39:07.942823 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":28,"thread_start_us":698,"threads_started":1}
I20260928 16:39:07.944694 22794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa5afa5368f45da87f539a6c3d39f7b" candidate_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f9df6bbe714a03ae43814617823f2a" is_pre_election: true
I20260928 16:39:07.944712 22938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa5afa5368f45da87f539a6c3d39f7b" candidate_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5777675a7ace4d72b8afef677e931ac2" is_pre_election: true
I20260928 16:39:07.945456 22794 raft_consensus.cc:2454] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6f92fdb2c9148e5bd37279c64c51e08 in term 0.
I20260928 16:39:07.945456 22938 raft_consensus.cc:2454] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6f92fdb2c9148e5bd37279c64c51e08 in term 0.
I20260928 16:39:07.946318 23006 leader_election.cc:302] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94f9df6bbe714a03ae43814617823f2a, d6f92fdb2c9148e5bd37279c64c51e08; no voters:
I20260928 16:39:07.946837 23142 raft_consensus.cc:2781] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:39:07.946995 23142 raft_consensus.cc:482] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:39:07.947094 23142 raft_consensus.cc:3037] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:39:07.950044 23142 raft_consensus.cc:504] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } }
I20260928 16:39:07.951094 23142 leader_election.cc:288] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [CANDIDATE]: Term 1 election: Requested vote from peers 5777675a7ace4d72b8afef677e931ac2 (127.10.247.194:34097), 94f9df6bbe714a03ae43814617823f2a (127.10.247.193:45751)
I20260928 16:39:07.951642 22794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa5afa5368f45da87f539a6c3d39f7b" candidate_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f9df6bbe714a03ae43814617823f2a"
I20260928 16:39:07.951632 22938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa5afa5368f45da87f539a6c3d39f7b" candidate_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5777675a7ace4d72b8afef677e931ac2"
I20260928 16:39:07.951879 22794 raft_consensus.cc:3037] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:39:07.951882 22938 raft_consensus.cc:3037] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:39:07.955478 22938 raft_consensus.cc:2454] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6f92fdb2c9148e5bd37279c64c51e08 in term 1.
I20260928 16:39:07.956004 22794 raft_consensus.cc:2454] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6f92fdb2c9148e5bd37279c64c51e08 in term 1.
I20260928 16:39:07.956049 23008 leader_election.cc:302] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5777675a7ace4d72b8afef677e931ac2, d6f92fdb2c9148e5bd37279c64c51e08; no voters:
I20260928 16:39:07.956522 23142 raft_consensus.cc:2781] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:39:07.957634 23142 raft_consensus.cc:686] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 1 LEADER]: Becoming Leader. State: Replica: d6f92fdb2c9148e5bd37279c64c51e08, State: Running, Role: LEADER
I20260928 16:39:07.958397 23142 consensus_queue.cc:237] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } }
I20260928 16:39:07.965529 22624 catalog_manager.cc:5495] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6f92fdb2c9148e5bd37279c64c51e08 (127.10.247.195). New cstate: current_term: 1 leader_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6f92fdb2c9148e5bd37279c64c51e08" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 } health_report { overall_health: UNKNOWN } } }
I20260928 16:39:08.007457 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:08.000892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.10.247.195:34135)
0928 16:39:08.001142 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:08.001169 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:08.001211 (+ 42us) server_negotiation.cc:407] Connection header received
0928 16:39:08.001506 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:08.001510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:08.001630 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:08.001791 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:39:08.002253 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:08.003098 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:08.004750 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:08.005273 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:08.005402 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:08.006038 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:39:08.006753 (+ 715us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:39:08.006912 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:08.007025 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 16:39:08.007131 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 16:39:08.018260 22938 raft_consensus.cc:1261] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [term 1 FOLLOWER]: Refusing update from remote peer d6f92fdb2c9148e5bd37279c64c51e08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:39:08.018261 22794 raft_consensus.cc:1261] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [term 1 FOLLOWER]: Refusing update from remote peer d6f92fdb2c9148e5bd37279c64c51e08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:39:08.019441 23146 consensus_queue.cc:1035] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94f9df6bbe714a03ae43814617823f2a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:39:08.019842 23142 consensus_queue.cc:1035] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5777675a7ace4d72b8afef677e931ac2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:39:08.034232 23149 mvcc.cc:205] Tried to move back new op lower bound from 7334353092669628416 to 7334353092443287552. Current Snapshot: MvccSnapshot[applied={T|T < 7334353092669628416}]
I20260928 16:39:08.034730 23150 mvcc.cc:205] Tried to move back new op lower bound from 7334353092669628416 to 7334353092443287552. Current Snapshot: MvccSnapshot[applied={T|T < 7334353092669628416}]
I20260928 16:39:08.055446 22625 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:39:08.055787 22625 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:39:08.060442 22625 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f35399965354bff93de65c195711db3: Sending DeleteTablet for 3 replicas of tablet 2aa5afa5368f45da87f539a6c3d39f7b
I20260928 16:39:08.062016 23052 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa5afa5368f45da87f539a6c3d39f7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:39:08 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:59088
I20260928 16:39:08.062021 22918 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa5afa5368f45da87f539a6c3d39f7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:39:08 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:32900
I20260928 16:39:08.062248 22774 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa5afa5368f45da87f539a6c3d39f7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:39:08 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:49368
I20260928 16:39:08.065972 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 22706
I20260928 16:39:08.074262 23161 tablet_replica.cc:330] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: stopping tablet replica
I20260928 16:39:08.075026 23161 raft_consensus.cc:2229] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:39:08.076043 23161 raft_consensus.cc:2258] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:39:08.079408 23161 ts_tablet_manager.cc:1905] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:39:08.080238 23159 tablet_replica.cc:330] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: stopping tablet replica
I20260928 16:39:08.080513 23160 tablet_replica.cc:330] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: stopping tablet replica
I20260928 16:39:08.080824 23159 raft_consensus.cc:2229] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:39:08.081003 23160 raft_consensus.cc:2229] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:39:08.081647 23160 raft_consensus.cc:2258] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:39:08.081647 23159 raft_consensus.cc:2258] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:39:08.084230 23159 ts_tablet_manager.cc:1905] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:39:08.089493 23160 ts_tablet_manager.cc:1905] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:39:08.090227 23161 ts_tablet_manager.cc:1918] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:39:08.090472 23161 log.cc:1196] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/2aa5afa5368f45da87f539a6c3d39f7b
I20260928 16:39:08.091238 23161 ts_tablet_manager.cc:1939] T 2aa5afa5368f45da87f539a6c3d39f7b P 94f9df6bbe714a03ae43814617823f2a: Deleting consensus metadata
I20260928 16:39:08.093655 22610 catalog_manager.cc:4841] TS 94f9df6bbe714a03ae43814617823f2a (127.10.247.193:45751): tablet 2aa5afa5368f45da87f539a6c3d39f7b (table test-table [id=91983168c17c410d9b31383f5e923583]) successfully deleted
I20260928 16:39:08.095542 23159 ts_tablet_manager.cc:1918] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:39:08.095873 23159 log.cc:1196] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/2aa5afa5368f45da87f539a6c3d39f7b
I20260928 16:39:08.097044 23159 ts_tablet_manager.cc:1939] T 2aa5afa5368f45da87f539a6c3d39f7b P d6f92fdb2c9148e5bd37279c64c51e08: Deleting consensus metadata
Sep 28 16:39:08 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:39:08.100131 22609 catalog_manager.cc:4841] TS d6f92fdb2c9148e5bd37279c64c51e08 (127.10.247.195:34135): tablet 2aa5afa5368f45da87f539a6c3d39f7b (table test-table [id=91983168c17c410d9b31383f5e923583]) successfully deleted
I20260928 16:39:08.102756 23160 ts_tablet_manager.cc:1918] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:39:08.102960 23160 log.cc:1196] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/2aa5afa5368f45da87f539a6c3d39f7b
I20260928 16:39:08.103504 23160 ts_tablet_manager.cc:1939] T 2aa5afa5368f45da87f539a6c3d39f7b P 5777675a7ace4d72b8afef677e931ac2: Deleting consensus metadata
I20260928 16:39:08.105582 22612 catalog_manager.cc:4841] TS 5777675a7ace4d72b8afef677e931ac2 (127.10.247.194:34097): tablet 2aa5afa5368f45da87f539a6c3d39f7b (table test-table [id=91983168c17c410d9b31383f5e923583]) successfully deleted
I20260928 16:39:08.109926 23135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:08.067700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49382 (local address 127.10.247.193:45751)
0928 16:39:08.067955 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:08.067963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:08.083813 (+ 15850us) server_negotiation.cc:407] Connection header received
0928 16:39:08.084059 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:08.084065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:08.084187 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:08.084316 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:39:08.086623 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:08.087859 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:08.090268 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:08.091010 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:08.091126 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:08.099730 (+ 8604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:39:08.099763 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:39:08.099788 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:39:08.099896 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:39:08.105890 (+ 5994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:08.106953 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:08.106965 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:08.106970 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:08.107158 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:08.107775 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:08.107781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:08.107784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:08.108415 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:39:08.108828 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:08.109185 (+ 357us) server_negotiation.cc:299] Negotiation successful
0928 16:39:08.109468 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260928 16:39:08.110857 22835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:39:08.263069 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 22706
I20260928 16:39:08.281210 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 22846
Sep 28 16:39:08 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:39:08.303397 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:08.282330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32912 (local address 127.10.247.194:34097)
0928 16:39:08.282537 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:08.282542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:08.282623 (+ 81us) server_negotiation.cc:407] Connection header received
0928 16:39:08.282859 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:08.282865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:08.282946 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:08.283039 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:39:08.285132 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:08.285911 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:08.288053 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:08.288518 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:08.288703 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:08.294922 (+ 6219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:39:08.294954 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:39:08.294982 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:39:08.295083 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:39:08.299449 (+ 4366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:08.300684 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:08.300697 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:08.300702 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:08.300925 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:08.301470 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:08.301476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:08.301479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:08.301966 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:39:08.302304 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:08.302679 (+ 375us) server_negotiation.cc:299] Negotiation successful
0928 16:39:08.302945 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260928 16:39:08.304246 22978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:39:08.402287 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 22846
I20260928 16:39:08.424883 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 22989
Sep 28 16:39:08 dist-test-slave-9fn6 krb5kdc[22144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:39:08.445650 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:39:08.425886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59102 (local address 127.10.247.195:34135)
0928 16:39:08.426128 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 16:39:08.426133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:39:08.426154 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:39:08.426361 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:39:08.426365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:39:08.426468 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:39:08.426555 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:39:08.429041 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:08.430364 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:08.432056 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:39:08.432898 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:39:08.433088 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:39:08.438339 (+ 5251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:39:08.438355 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:39:08.438375 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:39:08.438427 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:39:08.442005 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:08.442887 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:08.442896 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:08.442898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:08.443077 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:39:08.443768 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:39:08.443774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:39:08.443780 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:39:08.444258 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:39:08.444657 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0928 16:39:08.444990 (+ 333us) server_negotiation.cc:299] Negotiation successful
0928 16:39:08.445252 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260928 16:39:08.446434 23112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:39:08.593549 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 22989
I20260928 16:39:08.612556 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 22593
I20260928 16:39:08.614142 22655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:39:08.742650 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 22593
I20260928 16:39:08.768862 11231 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 16:39:09.116322 11231 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 16:39:09.116654 11231 mini_ranger.cc:66] Stopping Ranger...
I20260928 16:39:09.472931 11231 mini_ranger.cc:68] Stopped Ranger
2026-09-28 16:39:09.473 UTC [22174] LOG: received smart shutdown request
2026-09-28 16:39:09.478 UTC [22174] LOG: background worker "logical replication launcher" (PID 22184) exited with exit code 1
2026-09-28 16:39:09.478 UTC [22179] LOG: shutting down
2026-09-28 16:39:09.667 UTC [22174] LOG: database system is shut down
2026-09-28T16:39:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (55096 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:39:09 dist-test-slave-9fn6 krb5kdc[23176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:39:09 dist-test-slave-9fn6 krb5kdc[23176](info): set up 2 sockets
Sep 28 16:39:09 dist-test-slave-9fn6 krb5kdc[23176](info): commencing operation
krb5kdc: starting...
W20260928 16:39:10.833297 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:39:11 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:39:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:39:11Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.247.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.247.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/rangeradmin_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/rangeradmin_127.10.247.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.247.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.247.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/rangerlookup_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/rangerlookup_127.10.247.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.247.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.247.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/HTTP_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/HTTP_127.10.247.212@KRBTEST.COM.keytab.
I20260928 16:39:11.243889 11231 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKneNwT/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/postgres -l logfile start
2026-09-28 16:39:13.839 UTC [23206] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 16:39:13.839 UTC [23206] LOG: listening on IPv4 address "127.10.247.212", port 41491
2026-09-28 16:39:13.843 UTC [23206] LOG: listening on Unix socket "/tmp/.s.PGSQL.41491"
2026-09-28 16:39:13.866 UTC [23209] LOG: database system was shut down at 2026-09-28 16:39:12 UTC
2026-09-28 16:39:13.870 UTC [23206] LOG: database system is ready to accept connections
I20260928 16:39:14.772713 11231 mini_postgres.cc:96] Postgres bound to 41491
2026-09-28 16:39:14.795 UTC [23217] FATAL: database "slave" does not exist
127.10.247.212:41491 - accepting connections
I20260928 16:39:14.795840 11231 mini_ranger.cc:161] Starting Ranger...
I20260928 16:39:14.828756 11231 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 16:39:15.348520 11231 mini_ranger.cc:87] Created ranger Postgres database
I20260928 16:39:15.348695 11231 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-admin
2026-09-28 16:39:15,662 [I] DB FLAVOR :POSTGRES
2026-09-28 16:39:15,663 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 16:39:15,663 [I] Checking connection..
2026-09-28 16:39:15,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 16:39:15,953 [I] Checking connection passed.
2026-09-28 16:39:15,954 [I] --------- Verifying version history table ---------
2026-09-28 16:39:15,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 16:39:16,242 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 16:39:16,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 16:39:16,518 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 16:39:16,519 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 16:39:16,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 16:39:16.787 UTC [23281] WARNING: there is no transaction in progress
2026-09-28 16:39:16,794 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 16:39:16,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 16:39:17,065 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 16:39:17,066 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 16:39:17,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 16:39:17,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 16:39:17,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9fn6.c.gcp-upstream.internal','N') ;"
2026-09-28 16:39:17,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 16:39:18,149 [I] Table x_portal_user does not exist in database ranger
2026-09-28 16:39:18,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 16:39:18,415 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 16:39:18,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 16:39:18,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 16:39:18,939 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 16:39:18,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 16:39:20.029 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.048 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.068 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.086 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.103 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.123 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.189 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.198 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.216 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.228 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.239 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.256 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.266 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.274 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.282 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:20.288 UTC [23390] WARNING: there is no transaction in progress
2026-09-28 16:39:21,010 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 16:39:21,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 16:39:21,260 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 16:39:21,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 16:39:21,527 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 16:39:21,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 16:39:21,808 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 16:39:21,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 16:39:22,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 16:39:22,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9fn6.c.gcp-upstream.internal';"
2026-09-28 16:39:22,605 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 16:39:22,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 16:39:22,887 [I] DB_PATCHES have already been applied
I20260928 16:39:22.894682 11231 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKneNwT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 16:39:22.894795 11231 mini_ranger.cc:193] Using host: 127.10.247.212
I20260928 16:39:22.915114 11231 mini_ranger.cc:227] Ranger admin URL: http://127.10.247.212:58519
Sep 28, 2026 4:39:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 4:39:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 4:39:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 4:39:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 4:39:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 4:39:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 4:39:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 16:39:23.970534 11231 mini_ranger.cc:160] Time spent starting Ranger: real 9.175s user 0.005s sys 0.057s
Sep 28, 2026 4:39:47 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 4:39:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 4:39:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 4:39:47 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 4:39:48 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 16:39:49.494735 11231 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.247.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.247.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/rangerkms_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.247.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/rangerkms_127.10.247.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.247.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.247.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.247.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/HTTP_127.10.247.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.247.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/HTTP_127.10.247.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 16:39:49 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613589, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 16:39:49.742376 11231 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 16:39:49.772464 11231 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 16:39:50.696892 11231 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 16:39:50.697034 11231 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-kms
I20260928 16:39:50.697077 11231 mini_ranger_kms.cc:227] Using postgres at 127.10.247.212:41491
2026-09-28 16:39:51,119 [I] DB FLAVOR :POSTGRES
2026-09-28 16:39:51,119 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 16:39:51,120 [I] Checking connection
2026-09-28 16:39:51,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 16:39:51,374 [I] connection success
2026-09-28 16:39:51,375 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 16:39:51,375 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 16:39:51,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 16:39:51,645 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 16:39:51,645 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 16:39:51,646 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 16:39:51,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.247.212:41491/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 16:39:51,935 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 16:39:52.512511 11231 mini_ranger_kms.cc:310] Created kms service
I20260928 16:39:52.739578 11231 mini_ranger_kms.cc:326] Created kudu user
I20260928 16:39:52.791347 11231 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 16:39:52 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613589, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:39:53.267944 11231 mini_ranger_kms.cc:384] Added ranger policy
I20260928 16:39:53.268316 11231 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKneNwT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKneNwT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 16:39:53.268374 11231 mini_ranger_kms.cc:240] Using host: 127.10.247.212
I20260928 16:39:53.295288 11231 mini_ranger_kms.cc:276] Ranger KMS PID: 23572
I20260928 16:39:53.295433 11231 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.247.212:55169
Sep 28, 2026 4:39:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 4:39:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 4:39:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 4:39:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 4:39:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 4:39:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 16:39:55.490299 11231 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.748s user 0.073s sys 0.388s
I20260928 16:39:55.490618 11231 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 16:39:55.490828 11231 mini_ranger_kms.cc:401] 127.10.247.212:55169/kms/v1/keys
Sep 28 16:39:55 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790613595, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.247.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 16:39:56 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613595, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.247.212@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
Sep 28 16:39:56 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613595, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.247.212@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
Sep 28 16:39:59 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:39:59.553665 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37513
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37513
--ranger_config_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:55169/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:39:59.999142 23623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:39:59.999554 23623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:39:59.999681 23623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:00.008136 23623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:00.008301 23623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:00.008353 23623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:00.008431 23623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:00.008497 23623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:00.021354 23623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.247.212:55169/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37513
--ranger_config_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:00.023186 23623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:00.025024 23623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:00.041126 23628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:00.041373 23629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:00.041440 23631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:00.042261 23623 server_base.cc:1034] running on GCE node
I20260928 16:40:00.043325 23623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:00.045447 23623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:00.046690 23623 hybrid_clock.cc:648] HybridClock initialized: now 1790613600046609 us; error 68 us; skew 500 ppm
I20260928 16:40:00.047114 23623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:00 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613600, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:00.051496 23623 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:40:00.054015 23623 webserver.cc:460] Webserver started at http://127.10.247.254:34129/ using document root <none> and password file <none>
I20260928 16:40:00.055281 23623 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:00.055466 23623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:00.055985 23623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 16:40:00 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613600, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:00.119369 23623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "6e3641074a5c4601b1f123ecd6701a71"
format_stamp: "Formatted at 2026-09-28 16:40:00 on dist-test-slave-9fn6"
server_key: "119faae549057e5a99783c9869ad4f74"
server_key_iv: "08153095324ec96c3dc847fc3b3adfda"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:00.120684 23623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "6e3641074a5c4601b1f123ecd6701a71"
format_stamp: "Formatted at 2026-09-28 16:40:00 on dist-test-slave-9fn6"
server_key: "119faae549057e5a99783c9869ad4f74"
server_key_iv: "08153095324ec96c3dc847fc3b3adfda"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:00.129305 23623 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260928 16:40:00.156384 23638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:00.158644 23623 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260928 16:40:00.158905 23623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "6e3641074a5c4601b1f123ecd6701a71"
format_stamp: "Formatted at 2026-09-28 16:40:00 on dist-test-slave-9fn6"
server_key: "119faae549057e5a99783c9869ad4f74"
server_key_iv: "08153095324ec96c3dc847fc3b3adfda"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:00.159217 23623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:00.191322 23623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:00.197741 23623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:00.198175 23623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:00.222067 23689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:37513 every 8 connection(s)
I20260928 16:40:00.222091 23623 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:37513
I20260928 16:40:00.224514 23623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:40:00.229789 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 23623
I20260928 16:40:00.230077 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:40:00.230554 23690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:00.231073 11231 external_mini_cluster.cc:1428] Setting key 3bb580cf632f5470b35216b24387655e
I20260928 16:40:00.248575 23690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Bootstrap starting.
Sep 28 16:40:00 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:00.253398 23690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:00.254971 23690 log.cc:824] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:00.259670 23690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: No bootstrap required, opened a new log
I20260928 16:40:00.263551 23693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:00.234347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42494 (local address 127.10.247.254:37513)
0928 16:40:00.235414 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:00.235434 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:00.235517 (+ 83us) server_negotiation.cc:407] Connection header received
0928 16:40:00.237617 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:00.237665 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:00.238449 (+ 784us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:00.239117 (+ 668us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:00.241061 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:00.243148 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:00.245399 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:00.246709 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:00.246947 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:00.252773 (+ 5826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:00.252786 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:00.252812 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:00.252881 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:00.257282 (+ 4401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:00.258610 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:00.258621 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:00.258632 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:00.258887 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:00.259419 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:00.259423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:00.259426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:00.260301 (+ 875us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:00.260719 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:00.261271 (+ 552us) server_negotiation.cc:299] Negotiation successful
0928 16:40:00.261817 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":420,"threads_started":1}
I20260928 16:40:00.270553 23690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } }
I20260928 16:40:00.271226 23690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:00.271379 23690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e3641074a5c4601b1f123ecd6701a71, State: Initialized, Role: FOLLOWER
I20260928 16:40:00.272130 23690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } }
I20260928 16:40:00.272410 23690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:40:00.272552 23690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:40:00.272764 23690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:00.275449 23690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } }
I20260928 16:40:00.276253 23690 leader_election.cc:302] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e3641074a5c4601b1f123ecd6701a71; no voters:
I20260928 16:40:00.277124 23690 leader_election.cc:288] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:40:00.277284 23695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:40:00.278560 23695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 1 LEADER]: Becoming Leader. State: Replica: 6e3641074a5c4601b1f123ecd6701a71, State: Running, Role: LEADER
I20260928 16:40:00.279325 23695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } }
I20260928 16:40:00.279984 23690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:40:00.292100 23696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e3641074a5c4601b1f123ecd6701a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } } }
I20260928 16:40:00.292603 23696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:00.301920 23707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:40:00.301748 23697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e3641074a5c4601b1f123ecd6701a71. Latest consensus state: current_term: 1 leader_uuid: "6e3641074a5c4601b1f123ecd6701a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } } }
I20260928 16:40:00.302201 23697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:00.310083 23707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:40:00.311246 23690 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 16:40:00.340637 23707 catalog_manager.cc:1347] Generated new cluster ID: 9fc2b9b4fd1f4639802a1acbf517be3d
I20260928 16:40:00.340858 23707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:40:00.359135 23707 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:40:00.362053 23707 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:40:00.375746 23707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Generated new TSK 0
I20260928 16:40:00.377729 23707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:40:01.124590 23734 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:40:01.257320 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:55169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37513
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:40:01 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790613601, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 16:40:01.772904 23737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:01.773314 23737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:01.773455 23737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:01.781767 23737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:01.781919 23737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:01.782089 23737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:40:01.795475 23737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.247.212:55169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:01.798015 23737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:01.800364 23737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:01.826826 23747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:01.829416 23748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:01.832404 23737 server_base.cc:1034] running on GCE node
W20260928 16:40:01.833422 23750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:01.834498 23737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:01.836951 23737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:01.838320 23737 hybrid_clock.cc:648] HybridClock initialized: now 1790613601838192 us; error 138 us; skew 500 ppm
I20260928 16:40:01.838797 23737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:01 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613601, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:01.843968 23737 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:40:01.846578 23737 webserver.cc:460] Webserver started at http://127.10.247.193:36957/ using document root <none> and password file <none>
I20260928 16:40:01.847827 23737 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:01.847997 23737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:01.848464 23737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 16:40:01 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613601, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:01.872298 23737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "49aed82563d94280afb3e82c012ede18"
format_stamp: "Formatted at 2026-09-28 16:40:01 on dist-test-slave-9fn6"
server_key: "66494a30d565dbd94bed5f3c401b01ee"
server_key_iv: "d46df17503471f611fe25a0e53a5630c"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:01.874289 23737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "49aed82563d94280afb3e82c012ede18"
format_stamp: "Formatted at 2026-09-28 16:40:01 on dist-test-slave-9fn6"
server_key: "66494a30d565dbd94bed5f3c401b01ee"
server_key_iv: "d46df17503471f611fe25a0e53a5630c"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:01.891448 23737 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.007s sys 0.008s
I20260928 16:40:01.929641 23757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:01.938639 23737 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.007s sys 0.004s
I20260928 16:40:01.939291 23737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "49aed82563d94280afb3e82c012ede18"
format_stamp: "Formatted at 2026-09-28 16:40:01 on dist-test-slave-9fn6"
server_key: "66494a30d565dbd94bed5f3c401b01ee"
server_key_iv: "d46df17503471f611fe25a0e53a5630c"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:01.940069 23737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 28 16:40:01 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613601, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:02.087751 23737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:02.114782 23737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:02.115263 23737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:02.129343 23737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:02.141942 23737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:02.142124 23737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:02.142306 23737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:02.142378 23737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:02.348850 23869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:37469 every 8 connection(s)
I20260928 16:40:02.349313 23737 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:37469
I20260928 16:40:02.351397 23737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:40:02.361302 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 23737
I20260928 16:40:02.361608 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:40:02.362622 11231 external_mini_cluster.cc:1428] Setting key 4c63601aff4ff1f361c775166a312bc4
Sep 28 16:40:02 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613601, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:02.422180 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:02.361575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:52385 (local address 127.10.247.254:37513)
0928 16:40:02.381875 (+ 20300us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:02.381882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:02.381905 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:40:02.381974 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:02.381980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:02.382096 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:02.382233 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:02.392709 (+ 10476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:02.394030 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:02.397109 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:02.397813 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:02.397931 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:02.413367 (+ 15436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:02.413377 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:02.413382 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:02.413446 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:02.417285 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:02.418694 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:02.418701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:02.418704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:02.418895 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:02.419560 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:02.419566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:02.419570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:02.420034 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:02.420171 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:02.421453 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0928 16:40:02.421738 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20113,"thread_start_us":19922,"threads_started":1}
I20260928 16:40:02.424969 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:02.362378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37513 (local address 127.10.247.193:52385)
0928 16:40:02.376588 (+ 14210us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:02.376723 (+ 135us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:02.379537 (+ 2814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:02.390048 (+ 10511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:02.390069 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:02.391067 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:02.392293 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:02.392337 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:02.394349 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:02.394356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:02.396831 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:02.396867 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:02.398408 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:02.398415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:02.398773 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:02.409280 (+ 10507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:02.409332 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:02.412894 (+ 3562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:02.417605 (+ 4711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:02.417617 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:02.417645 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:02.418505 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:02.419063 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:02.419068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:02.419072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:02.419408 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:02.420291 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:02.420301 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:02.420934 (+ 633us) client_negotiation.cc:769] Sending connection context
0928 16:40:02.422473 (+ 1539us) client_negotiation.cc:240] Negotiation successful
0928 16:40:02.423181 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13732,"thread_start_us":17658,"threads_started":1}
I20260928 16:40:02.427373 23870 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37513
I20260928 16:40:02.427784 23870 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:02.428946 23870 heartbeater.cc:511] Master 127.10.247.254:37513 requested a full tablet report, sending...
I20260928 16:40:02.435504 23655 ts_manager.cc:194] Registered new tserver with Master: 49aed82563d94280afb3e82c012ede18 (127.10.247.193:37469)
I20260928 16:40:02.437978 23655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:52385
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Sep 28 16:40:02 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613601, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
W20260928 16:40:02.508111 11231 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.247.194: real 0.124s user 0.002s sys 0.025s
I20260928 16:40:02.509239 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:55169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37513
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:03.108994 23877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:03.109565 23877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:03.109730 23877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:03.121248 23877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:03.121447 23877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:03.121649 23877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:40:03.138150 23877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.247.212:55169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:03.140516 23877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:03.142748 23877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:03.163926 23891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:03.164733 23890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:03.168881 23877 server_base.cc:1034] running on GCE node
W20260928 16:40:03.169416 23893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:03.170430 23877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:03.172008 23877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:03.173322 23877 hybrid_clock.cc:648] HybridClock initialized: now 1790613603173240 us; error 72 us; skew 500 ppm
I20260928 16:40:03.173790 23877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:03 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:03.179026 23877 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:40:03.181890 23877 webserver.cc:460] Webserver started at http://127.10.247.194:46005/ using document root <none> and password file <none>
I20260928 16:40:03.183130 23877 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:03.183300 23877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:03.183763 23877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 16:40:03 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:03.212234 23877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "f80bd7b07d184718aa55c3d21c607ad2"
format_stamp: "Formatted at 2026-09-28 16:40:03 on dist-test-slave-9fn6"
server_key: "aec437f9068a73d6e210aede126d745b"
server_key_iv: "5eeb71c3f602f7dc47db3a8c444cb3fc"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:03.213352 23877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "f80bd7b07d184718aa55c3d21c607ad2"
format_stamp: "Formatted at 2026-09-28 16:40:03 on dist-test-slave-9fn6"
server_key: "aec437f9068a73d6e210aede126d745b"
server_key_iv: "5eeb71c3f602f7dc47db3a8c444cb3fc"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:03.220759 23877 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260928 16:40:03.245414 23900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:03.247494 23877 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260928 16:40:03.247757 23877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "f80bd7b07d184718aa55c3d21c607ad2"
format_stamp: "Formatted at 2026-09-28 16:40:03 on dist-test-slave-9fn6"
server_key: "aec437f9068a73d6e210aede126d745b"
server_key_iv: "5eeb71c3f602f7dc47db3a8c444cb3fc"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:03.248044 23877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:03.283368 23877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:03.288898 23877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:03.289255 23877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:03.290696 23877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:03.292501 23877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:03.292637 23877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:03.292778 23877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:03.292832 23877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:03.333918 23877 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:41829
I20260928 16:40:03.333925 24012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:41829 every 8 connection(s)
I20260928 16:40:03.335840 23877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:40:03.341756 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 23877
I20260928 16:40:03.341990 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:40:03.342763 11231 external_mini_cluster.cc:1428] Setting key 84ee1dd32ca059fcc83a84f438475e71
Sep 28 16:40:03 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
I20260928 16:40:03.382138 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:03.340143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:55447 (local address 127.10.247.254:37513)
0928 16:40:03.359409 (+ 19266us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:03.359417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:03.359455 (+ 38us) server_negotiation.cc:407] Connection header received
0928 16:40:03.359930 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:03.359936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:03.360060 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:03.360209 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:03.362912 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:03.364099 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:03.366810 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:03.367411 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:03.367554 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:03.372818 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:03.372827 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:03.372832 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:03.372891 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:03.377192 (+ 4301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:03.378561 (+ 1369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:03.378570 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:03.378574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:03.378795 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:03.379538 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:03.379544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:03.379548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:03.380045 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:03.380249 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:03.381519 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0928 16:40:03.381779 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19066,"thread_start_us":19294,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
I20260928 16:40:03.383934 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:03.341065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37513 (local address 127.10.247.194:55447)
0928 16:40:03.354783 (+ 13718us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:03.354884 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:03.359441 (+ 4557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:03.360586 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:03.360603 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:03.361454 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:03.362577 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:03.362612 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:03.364320 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:03.364326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:03.366605 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:03.366639 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:03.367690 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:03.367697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:03.368042 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:03.369058 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:03.369096 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:03.372347 (+ 3251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:03.377467 (+ 5120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:03.377482 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:03.377512 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:03.378344 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:03.378944 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:03.378950 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:03.378954 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:03.379361 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:03.380220 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:03.380231 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:03.380945 (+ 714us) client_negotiation.cc:769] Sending connection context
0928 16:40:03.381353 (+ 408us) client_negotiation.cc:240] Negotiation successful
0928 16:40:03.382124 (+ 771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13198,"thread_start_us":14130,"threads_started":1}
I20260928 16:40:03.386281 24013 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37513
I20260928 16:40:03.386749 24013 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:03.387960 24013 heartbeater.cc:511] Master 127.10.247.254:37513 requested a full tablet report, sending...
I20260928 16:40:03.390652 23655 ts_manager.cc:194] Registered new tserver with Master: f80bd7b07d184718aa55c3d21c607ad2 (127.10.247.194:41829)
I20260928 16:40:03.391940 23655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:55447
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:40:03.420266 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:55169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37513
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:40:03.443600 23870 heartbeater.cc:503] Master 127.10.247.254:37513 was elected leader, sending a full tablet report...
W20260928 16:40:03.846958 24019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:03.847342 24019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:03.847447 24019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:03.855571 24019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:03.855762 24019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:03.855912 24019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:40:03.868073 24019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.247.212:55169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:03.869927 24019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:03.871748 24019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:03.887833 24025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:03.887867 24024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:03.887887 24027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:03.889521 24019 server_base.cc:1034] running on GCE node
I20260928 16:40:03.890414 24019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:03.891811 24019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:03.893074 24019 hybrid_clock.cc:648] HybridClock initialized: now 1790613603893009 us; error 52 us; skew 500 ppm
I20260928 16:40:03.893517 24019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:03 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:03.897421 24019 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:40:03.899346 24019 webserver.cc:460] Webserver started at http://127.10.247.195:36927/ using document root <none> and password file <none>
I20260928 16:40:03.900252 24019 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:03.900389 24019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:03.900810 24019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 16:40:03 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:03.919281 24019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "740607d98bce48d8a8de456a31373498"
format_stamp: "Formatted at 2026-09-28 16:40:03 on dist-test-slave-9fn6"
server_key: "cd94e6b251a1cf9fc192f71ebd47f7d7"
server_key_iv: "b1f17f7f1dede4c39c8e9d4f39eac8f1"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:03.920375 24019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "740607d98bce48d8a8de456a31373498"
format_stamp: "Formatted at 2026-09-28 16:40:03 on dist-test-slave-9fn6"
server_key: "cd94e6b251a1cf9fc192f71ebd47f7d7"
server_key_iv: "b1f17f7f1dede4c39c8e9d4f39eac8f1"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:03.927340 24019 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:40:03.952543 24034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:03.954974 24019 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260928 16:40:03.955256 24019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "740607d98bce48d8a8de456a31373498"
format_stamp: "Formatted at 2026-09-28 16:40:03 on dist-test-slave-9fn6"
server_key: "cd94e6b251a1cf9fc192f71ebd47f7d7"
server_key_iv: "b1f17f7f1dede4c39c8e9d4f39eac8f1"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:03.955605 24019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:04.003476 24019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:04.008714 24019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:04.009032 24019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:04.010269 24019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:04.011960 24019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:04.012042 24019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:04.012138 24019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:04.012192 24019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:04.047331 24019 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:37563
I20260928 16:40:04.047367 24146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:37563 every 8 connection(s)
I20260928 16:40:04.049115 24019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:40:04.051456 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 24019
I20260928 16:40:04.051832 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:40:04.052454 11231 external_mini_cluster.cc:1428] Setting key e7becc987b8be5b5ebb8dd34976dddfd
Sep 28 16:40:04 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:04.073992 24149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:04.052698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:46473 (local address 127.10.247.254:37513)
0928 16:40:04.053362 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:04.053368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:04.054813 (+ 1445us) server_negotiation.cc:407] Connection header received
0928 16:40:04.057406 (+ 2593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:04.057413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:04.057538 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:04.057649 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:04.059604 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.060467 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.062393 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.062902 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.063023 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:04.066970 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:04.066975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:04.066979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:04.067024 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:04.070414 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:04.071267 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:04.071274 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:04.071278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:04.071423 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:04.071894 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:04.071899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:04.071901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:04.072238 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:04.072384 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:04.073325 (+ 941us) server_negotiation.cc:299] Negotiation successful
0928 16:40:04.073598 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":284,"threads_started":1}
I20260928 16:40:04.075227 24150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:04.053551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37513 (local address 127.10.247.195:46473)
0928 16:40:04.054631 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:04.054683 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:04.057078 (+ 2395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:04.057984 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:04.057995 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:04.058546 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:04.059391 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:04.059417 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.060696 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.060700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:04.062206 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:04.062250 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.063165 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.063174 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:04.063448 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:04.064212 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:04.064247 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:04.066708 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:04.070568 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:04.070590 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:04.070610 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:04.071136 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:04.071531 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:04.071537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:04.071541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:04.071774 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:04.072363 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:04.072373 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:04.072894 (+ 521us) client_negotiation.cc:769] Sending connection context
0928 16:40:04.073177 (+ 283us) client_negotiation.cc:240] Negotiation successful
0928 16:40:04.073850 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":474,"threads_started":1}
I20260928 16:40:04.076865 24147 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37513
I20260928 16:40:04.077240 24147 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:04.078126 24147 heartbeater.cc:511] Master 127.10.247.254:37513 requested a full tablet report, sending...
I20260928 16:40:04.079856 23655 ts_manager.cc:194] Registered new tserver with Master: 740607d98bce48d8a8de456a31373498 (127.10.247.195:37563)
I20260928 16:40:04.080926 23655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:46473
I20260928 16:40:04.082705 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:40:04.084666 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 23737
Sep 28 16:40:04 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:40:04.102823 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:04.085609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53726 (local address 127.10.247.193:37469)
0928 16:40:04.086225 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:04.086232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:04.086282 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:40:04.086352 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:04.086359 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:04.086605 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:04.086742 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:04.088372 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.089669 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.091198 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.091824 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.091930 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:04.096794 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:04.096813 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:04.096833 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:04.096879 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:04.099978 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:04.100777 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:04.100785 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:04.100788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:04.100950 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:04.101303 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:04.101306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:04.101308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:04.101713 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:04.101987 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:04.102252 (+ 265us) server_negotiation.cc:299] Negotiation successful
0928 16:40:04.102500 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":268,"threads_started":1}
I20260928 16:40:04.104398 23865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:04.194300 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 23737
I20260928 16:40:04.209772 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 23877
Sep 28 16:40:04 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:40:04.234220 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:04.210844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32932 (local address 127.10.247.194:41829)
0928 16:40:04.211466 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:04.211474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:04.211507 (+ 33us) server_negotiation.cc:407] Connection header received
0928 16:40:04.211588 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:04.211596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:04.211870 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:04.212034 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:04.213536 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.214684 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.216276 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.217005 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.217139 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:04.227213 (+ 10074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:04.227236 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:04.227259 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:04.227320 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:04.230675 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:04.231602 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:04.231612 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:04.231615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:04.231806 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:04.232232 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:04.232237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:04.232239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:04.232763 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:04.233110 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:04.233484 (+ 374us) server_negotiation.cc:299] Negotiation successful
0928 16:40:04.233818 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":293,"threads_started":1}
I20260928 16:40:04.235810 24008 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:04.324560 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 23877
I20260928 16:40:04.339195 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 24019
Sep 28 16:40:04 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:40:04.358889 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:04.340355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.10.247.195:37563)
0928 16:40:04.341077 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:04.341085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:04.341121 (+ 36us) server_negotiation.cc:407] Connection header received
0928 16:40:04.341202 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:04.341211 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:04.341496 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:04.341681 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:04.343220 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.344424 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.346019 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:04.346793 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:04.346933 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:04.351873 (+ 4940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:04.351893 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:04.351914 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:04.351965 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:04.355372 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:04.356315 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:04.356325 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:04.356327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:04.356496 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:04.356981 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:04.356986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:04.356989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:04.357426 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:04.357809 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:04.358133 (+ 324us) server_negotiation.cc:299] Negotiation successful
0928 16:40:04.358455 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":331,"threads_started":1}
I20260928 16:40:04.360602 24142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:04.451602 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 24019
I20260928 16:40:04.466642 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 23623
I20260928 16:40:04.468250 23685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:04.589875 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 23623
I20260928 16:40:04.611150 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37513
--webserver_interface=127.10.247.254
--webserver_port=34129
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37513
--ranger_config_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:55169/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.247.212:55169/kms with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:05.017969 24159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:05.018429 24159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:05.018590 24159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:05.026806 24159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:05.026985 24159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:05.027073 24159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:05.027163 24159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:05.027233 24159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:05.039258 24159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.247.212:55169/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37513
--ranger_config_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=34129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:05.041121 24159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:05.042833 24159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:05.057118 24164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:05.057209 24167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:05.057154 24165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:05.058110 24159 server_base.cc:1034] running on GCE node
I20260928 16:40:05.059118 24159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:05.061146 24159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:05.062392 24159 hybrid_clock.cc:648] HybridClock initialized: now 1790613605062332 us; error 55 us; skew 500 ppm
I20260928 16:40:05.062826 24159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:05 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613605, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:05.066424 24159 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:40:05.068301 24159 webserver.cc:460] Webserver started at http://127.10.247.254:34129/ using document root <none> and password file <none>
I20260928 16:40:05.069188 24159 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:05.069337 24159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 16:40:05 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613605, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:05.151072 24159 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:40:05.155107 24175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:05.157051 24159 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:40:05.157308 24159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "6e3641074a5c4601b1f123ecd6701a71"
format_stamp: "Formatted at 2026-09-28 16:40:00 on dist-test-slave-9fn6"
server_key: "119faae549057e5a99783c9869ad4f74"
server_key_iv: "08153095324ec96c3dc847fc3b3adfda"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:05.158545 24159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:05.197577 24159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:05.202944 24159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:05.203298 24159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:05.224229 24159 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:37513
I20260928 16:40:05.224273 24226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:37513 every 8 connection(s)
I20260928 16:40:05.226236 24159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:40:05.235152 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 24159
I20260928 16:40:05.235648 24227 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:40:05.236479 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:55169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:37469
--local_ip_for_outbound_sockets=127.10.247.193
--tserver_master_addrs=127.10.247.254:37513
--webserver_port=36957
--webserver_interface=127.10.247.193
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.247.212:55169/kms with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:40:05.241413 24227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Bootstrap starting.
I20260928 16:40:05.267350 24227 log.cc:824] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:05.279726 24227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:40:05.280437 24227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Bootstrap complete.
I20260928 16:40:05.290949 24227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } }
I20260928 16:40:05.292030 24227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e3641074a5c4601b1f123ecd6701a71, State: Initialized, Role: FOLLOWER
I20260928 16:40:05.292946 24227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } }
I20260928 16:40:05.293234 24227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:40:05.293365 24227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:40:05.293546 24227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 1 FOLLOWER]: Advancing to term 2
I20260928 16:40:05.298362 24227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } }
I20260928 16:40:05.299042 24227 leader_election.cc:302] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e3641074a5c4601b1f123ecd6701a71; no voters:
I20260928 16:40:05.299759 24227 leader_election.cc:288] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 16:40:05.299935 24231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 16:40:05.301462 24231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [term 2 LEADER]: Becoming Leader. State: Replica: 6e3641074a5c4601b1f123ecd6701a71, State: Running, Role: LEADER
I20260928 16:40:05.302296 24231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } }
I20260928 16:40:05.302681 24227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:40:05.305526 24233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e3641074a5c4601b1f123ecd6701a71. Latest consensus state: current_term: 2 leader_uuid: "6e3641074a5c4601b1f123ecd6701a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } } }
I20260928 16:40:05.305987 24233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:05.305886 24232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6e3641074a5c4601b1f123ecd6701a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3641074a5c4601b1f123ecd6701a71" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37513 } } }
I20260928 16:40:05.306272 24232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71 [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:05.317593 24240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:40:05.324946 24240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:40:05.326005 24227 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 16:40:05.326876 24240 catalog_manager.cc:1259] Loaded cluster ID: 9fc2b9b4fd1f4639802a1acbf517be3d
I20260928 16:40:05.326977 24240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:40:05.331624 24240 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:40:05.333806 24240 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Loaded TSK: 0
I20260928 16:40:05.335754 24240 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 16:40:05.764637 24229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:05.765177 24229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:05.765339 24229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:05.777853 24229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:05.778079 24229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:05.778316 24229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:40:05.795136 24229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.247.212:55169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:37469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=36957
--tserver_master_addrs=127.10.247.254:37513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:05.797613 24229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:05.800009 24229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:05.841437 24265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:05.846810 24267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:05.849956 24264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:05.851725 24229 server_base.cc:1034] running on GCE node
I20260928 16:40:05.857203 24229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:05.859587 24229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:05.862984 24229 hybrid_clock.cc:648] HybridClock initialized: now 1790613605862897 us; error 103 us; skew 500 ppm
I20260928 16:40:05.863579 24229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:05 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613605, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:05.869020 24229 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:40:05.878186 24229 webserver.cc:460] Webserver started at http://127.10.247.193:36957/ using document root <none> and password file <none>
I20260928 16:40:05.879169 24229 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:05.879351 24229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 16:40:05 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613605, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:05.909608 24229 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:40:05.914386 24275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:05.916746 24229 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260928 16:40:05.916985 24229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "49aed82563d94280afb3e82c012ede18"
format_stamp: "Formatted at 2026-09-28 16:40:01 on dist-test-slave-9fn6"
server_key: "66494a30d565dbd94bed5f3c401b01ee"
server_key_iv: "d46df17503471f611fe25a0e53a5630c"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:05.918932 24229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:05.953397 24229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:05.959192 24229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:05.959515 24229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:05.960821 24229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:05.962711 24229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:05.962812 24229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:05.962924 24229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:05.962993 24229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:06.044824 24229 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:37469
I20260928 16:40:06.044824 24388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:37469 every 8 connection(s)
I20260928 16:40:06.047494 24229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:40:06.054991 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 24229
I20260928 16:40:06.056107 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:55169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:41829
--local_ip_for_outbound_sockets=127.10.247.194
--tserver_master_addrs=127.10.247.254:37513
--webserver_port=46005
--webserver_interface=127.10.247.194
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.247.212:55169/kms with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:40:06 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613605, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:06.104671 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:06.052188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37513 (local address 127.10.247.193:41503)
0928 16:40:06.068313 (+ 16125us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:06.068407 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:06.070690 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:06.073984 (+ 3294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:06.073997 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:06.074894 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:06.076026 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:06.076068 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:06.079061 (+ 2993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:06.079068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:06.081203 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:06.081226 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:06.082937 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:06.082947 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:06.083285 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:06.084243 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:06.084281 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:06.087722 (+ 3441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:06.093868 (+ 6146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:06.093883 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:06.093907 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:06.095218 (+ 1311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:06.095999 (+ 781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:06.096006 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:06.096011 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:06.096372 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:06.098432 (+ 2060us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:06.098444 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:06.099087 (+ 643us) client_negotiation.cc:769] Sending connection context
0928 16:40:06.099510 (+ 423us) client_negotiation.cc:240] Negotiation successful
0928 16:40:06.100333 (+ 823us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15709,"thread_start_us":15459,"threads_started":1}
I20260928 16:40:06.105692 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:06.051936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:41503 (local address 127.10.247.254:37513)
0928 16:40:06.067433 (+ 15497us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:06.067467 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:06.071449 (+ 3982us) server_negotiation.cc:407] Connection header received
0928 16:40:06.071919 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:06.071934 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:06.072437 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:06.072961 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:06.076447 (+ 3486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:06.078775 (+ 2328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:06.081482 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:06.082588 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:06.082828 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:06.088245 (+ 5417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:06.088263 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:06.088294 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:06.088383 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:06.093536 (+ 5153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:06.095485 (+ 1949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:06.095504 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:06.095512 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:06.095758 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:06.096587 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:06.096593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:06.096597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:06.097991 (+ 1394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:06.098279 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:06.101065 (+ 2786us) server_negotiation.cc:299] Negotiation successful
0928 16:40:06.101644 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14966,"thread_start_us":19017,"threads_started":1}
I20260928 16:40:06.110253 24389 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37513
I20260928 16:40:06.110699 24389 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:06.111867 24389 heartbeater.cc:511] Master 127.10.247.254:37513 requested a full tablet report, sending...
W20260928 16:40:06.116345 24389 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:37513 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 16:40:06.116747 24389 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:06.117146 24389 heartbeater.cc:511] Master 127.10.247.254:37513 requested a full tablet report, sending...
I20260928 16:40:06.118783 24389 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:06.119189 24389 heartbeater.cc:511] Master 127.10.247.254:37513 requested a full tablet report, sending...
W20260928 16:40:06.120973 24389 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 16:40:06.540262 24393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:06.540702 24393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:06.540874 24393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:06.549437 24393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:06.549605 24393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:06.549844 24393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:40:06.561729 24393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.247.212:55169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:41829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=46005
--tserver_master_addrs=127.10.247.254:37513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:06.563630 24393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:06.565706 24393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:06.585274 24407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:06.590868 24408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:06.609546 24393 server_base.cc:1034] running on GCE node
W20260928 16:40:06.611495 24410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:06.612703 24393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:06.614934 24393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:06.616482 24393 hybrid_clock.cc:648] HybridClock initialized: now 1790613606616383 us; error 91 us; skew 500 ppm
I20260928 16:40:06.617065 24393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:06 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:06.622530 24393 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:40:06.652271 24393 webserver.cc:460] Webserver started at http://127.10.247.194:46005/ using document root <none> and password file <none>
I20260928 16:40:06.653468 24393 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:06.653671 24393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 16:40:06 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:06.690387 24393 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.007s sys 0.000s
I20260928 16:40:06.694193 24420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:06.696249 24393 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:40:06.696461 24393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "f80bd7b07d184718aa55c3d21c607ad2"
format_stamp: "Formatted at 2026-09-28 16:40:03 on dist-test-slave-9fn6"
server_key: "aec437f9068a73d6e210aede126d745b"
server_key_iv: "5eeb71c3f602f7dc47db3a8c444cb3fc"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:06.697814 24393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:06.718686 24393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
Sep 28 16:40:06 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790613606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:06.723683 24393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:06.724000 24393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:06.725528 24393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:06.727525 24393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:06.727640 24393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:06.727771 24393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:06.727845 24393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:06.789381 24393 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:41829
I20260928 16:40:06.789464 24532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:41829 every 8 connection(s)
I20260928 16:40:06.791750 24393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:40:06.797772 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 24393
I20260928 16:40:06.798707 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.247.212:55169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:37563
--local_ip_for_outbound_sockets=127.10.247.195
--tserver_master_addrs=127.10.247.254:37513
--webserver_port=36927
--webserver_interface=127.10.247.195
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.247.212:55169/kms with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:40:06 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:06.830163 24535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:06.795626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:58651 (local address 127.10.247.254:37513)
0928 16:40:06.796216 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:06.796222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:06.797899 (+ 1677us) server_negotiation.cc:407] Connection header received
0928 16:40:06.804333 (+ 6434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:06.804342 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:06.804510 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:06.804730 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:06.809380 (+ 4650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:06.810780 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:06.814031 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:06.814739 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:06.814897 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:06.820319 (+ 5422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:06.820329 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:06.820335 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:06.820395 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:06.824267 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:06.825733 (+ 1466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:06.825741 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:06.825745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:06.825987 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:06.826732 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:06.826739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:06.826742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:06.827216 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:06.827401 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:06.829325 (+ 1924us) server_negotiation.cc:299] Negotiation successful
0928 16:40:06.829678 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":285,"threads_started":1}
I20260928 16:40:06.833431 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:06.796511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37513 (local address 127.10.247.194:58651)
0928 16:40:06.797650 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:06.797714 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:06.803731 (+ 6017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:06.805325 (+ 1594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:06.805343 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:06.807523 (+ 2180us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:06.808983 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:06.809024 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:06.811104 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:06.811112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:06.813745 (+ 2633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:06.813772 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:06.815083 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:06.815094 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:06.815482 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:06.816478 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:06.816521 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:06.819882 (+ 3361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:06.824547 (+ 4665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:06.824560 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:06.824589 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:06.825487 (+ 898us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:06.826174 (+ 687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:06.826179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:06.826183 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:06.826559 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:06.827493 (+ 934us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:06.827505 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:06.828233 (+ 728us) client_negotiation.cc:769] Sending connection context
0928 16:40:06.828730 (+ 497us) client_negotiation.cc:240] Negotiation successful
0928 16:40:06.831414 (+ 2684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":392,"threads_started":1}
I20260928 16:40:06.836836 24533 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37513
I20260928 16:40:06.837332 24533 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:06.838611 24533 heartbeater.cc:511] Master 127.10.247.254:37513 requested a full tablet report, sending...
I20260928 16:40:06.841941 24192 ts_manager.cc:194] Registered new tserver with Master: f80bd7b07d184718aa55c3d21c607ad2 (127.10.247.194:41829)
I20260928 16:40:06.844769 24192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:58651
I20260928 16:40:07.121249 24389 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:07.121897 24389 heartbeater.cc:511] Master 127.10.247.254:37513 requested a full tablet report, sending...
I20260928 16:40:07.123872 24192 ts_manager.cc:194] Registered new tserver with Master: 49aed82563d94280afb3e82c012ede18 (127.10.247.193:37469)
I20260928 16:40:07.125576 24192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:41503
Sep 28 16:40:07 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
W20260928 16:40:07.341935 24538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:07.342532 24538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:07.342705 24538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:07.354123 24538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:07.354313 24538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:07.354511 24538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:40:07.371157 24538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.247.212:55169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:37563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=36927
--tserver_master_addrs=127.10.247.254:37513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:07.373494 24538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:07.375603 24538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:07.395838 24543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:07.396473 24544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:07.398226 24546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:07.399603 24538 server_base.cc:1034] running on GCE node
I20260928 16:40:07.400592 24538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:07.402181 24538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:07.403460 24538 hybrid_clock.cc:648] HybridClock initialized: now 1790613607403347 us; error 97 us; skew 500 ppm
I20260928 16:40:07.404004 24538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:07 dist-test-slave-9fn6 krb5kdc[23176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613607, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:07.409551 24538 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:40:07.414760 24538 webserver.cc:460] Webserver started at http://127.10.247.195:36927/ using document root <none> and password file <none>
I20260928 16:40:07.416043 24538 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:07.416239 24538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 16:40:07 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613607, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:07.447914 24538 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.005s sys 0.000s
I20260928 16:40:07.453174 24554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:07.455765 24538 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260928 16:40:07.456087 24538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "740607d98bce48d8a8de456a31373498"
format_stamp: "Formatted at 2026-09-28 16:40:03 on dist-test-slave-9fn6"
server_key: "cd94e6b251a1cf9fc192f71ebd47f7d7"
server_key_iv: "b1f17f7f1dede4c39c8e9d4f39eac8f1"
server_key_version: "kuduclusterkey@0"
I20260928 16:40:07.457975 24538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:07.488859 24538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:07.495222 24538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:07.495640 24538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:07.497254 24538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:07.500998 24538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:07.501168 24538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:07.501327 24538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:07.501411 24538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:40:07 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for HTTP/127.10.247.212@KRBTEST.COM
I20260928 16:40:07.851409 24533 heartbeater.cc:503] Master 127.10.247.254:37513 was elected leader, sending a full tablet report...
I20260928 16:40:07.865334 24538 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:37563
I20260928 16:40:07.865330 24668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:37563 every 8 connection(s)
I20260928 16:40:07.868044 24538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:40:07.877903 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 24538
Sep 28 16:40:07 dist-test-slave-9fn6 krb5kdc[23176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613607, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:07.907868 24672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.874492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:33049 (local address 127.10.247.254:37513)
0928 16:40:07.881047 (+ 6555us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:07.881055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:07.881083 (+ 28us) server_negotiation.cc:407] Connection header received
0928 16:40:07.881169 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:07.881175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:07.881293 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:07.881410 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:07.884519 (+ 3109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:07.885855 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:07.888993 (+ 3138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:07.889870 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:07.889987 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:07.897043 (+ 7056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:07.897051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:07.897057 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:07.897155 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:07.902733 (+ 5578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:07.904237 (+ 1504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:07.904245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:07.904249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:07.904476 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:07.905240 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:07.905251 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:07.905254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:07.905744 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:07.905909 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:07.907081 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0928 16:40:07.907365 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6390,"thread_start_us":6757,"threads_started":1}
I20260928 16:40:07.909416 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.872916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37513 (local address 127.10.247.195:33049)
0928 16:40:07.877341 (+ 4425us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:07.877418 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:07.880139 (+ 2721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:07.882182 (+ 2043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:07.882198 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:07.883069 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:07.884189 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:07.884227 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:07.886157 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:07.886163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:07.888732 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:07.888758 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:07.890478 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:07.890485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:07.890861 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:07.893233 (+ 2372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:07.893274 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:07.896702 (+ 3428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:07.903043 (+ 6341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:07.903060 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:07.903091 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:07.904011 (+ 920us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:07.904695 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:07.904701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:07.904704 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:07.905061 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:07.906022 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:07.906033 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:07.906589 (+ 556us) client_negotiation.cc:769] Sending connection context
0928 16:40:07.906917 (+ 328us) client_negotiation.cc:240] Negotiation successful
0928 16:40:07.907687 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4014,"thread_start_us":3717,"threads_started":1}
I20260928 16:40:07.911708 24669 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37513
I20260928 16:40:07.912175 24669 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:07.913430 24669 heartbeater.cc:511] Master 127.10.247.254:37513 requested a full tablet report, sending...
I20260928 16:40:07.914405 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.879776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.10.247.254:37513)
0928 16:40:07.886083 (+ 6307us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:07.886090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:07.886108 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:40:07.886164 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:07.886169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:07.886290 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:07.886425 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:07.889217 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:07.890552 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:07.895681 (+ 5129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:07.896672 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:07.896909 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:07.901892 (+ 4983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:07.901904 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:07.901910 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:07.902002 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:07.906091 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:07.908415 (+ 2324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:07.908421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:07.908425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:07.908597 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:07.911020 (+ 2423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:07.911027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:07.911030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:07.911469 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:07.911806 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:07.913633 (+ 1827us) server_negotiation.cc:299] Negotiation successful
0928 16:40:07.913936 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6143,"spinlock_wait_cycles":6912,"thread_start_us":9697,"threads_started":1}
I20260928 16:40:07.915740 24192 ts_manager.cc:194] Registered new tserver with Master: 740607d98bce48d8a8de456a31373498 (127.10.247.195:37563)
I20260928 16:40:07.917192 24192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:33049
I20260928 16:40:07.918320 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:40:07.942981 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.923805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50572 (local address 127.10.247.254:37513)
0928 16:40:07.924034 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:07.924042 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:07.924599 (+ 557us) server_negotiation.cc:407] Connection header received
0928 16:40:07.924895 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:07.924901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:07.925016 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:07.925134 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:07.927414 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:07.928553 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:07.930698 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:07.931406 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:07.931528 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:07.935661 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:07.935669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:07.935673 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:07.935735 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:07.939803 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:07.940892 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:07.940898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:07.940900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:07.941071 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:07.941488 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:07.941491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:07.941493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:07.941819 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:07.942029 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:07.942452 (+ 423us) server_negotiation.cc:299] Negotiation successful
0928 16:40:07.942675 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 16:40:07.949419 24191 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:40:07.958392 24191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:40:08.019069 24695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.977531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:41829 (local address 127.0.0.1:32942)
0928 16:40:08.004662 (+ 27131us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.004744 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:08.005091 (+ 347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:08.007447 (+ 2356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:08.007460 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:08.007513 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:08.008111 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.008129 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.011883 (+ 3754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.011896 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.014716 (+ 2820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.014737 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.017696 (+ 2959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.017704 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.017969 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.018037 (+ 68us) client_negotiation.cc:769] Sending connection context
0928 16:40:08.018224 (+ 187us) client_negotiation.cc:240] Negotiation successful
0928 16:40:08.018567 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25393,"thread_start_us":30945,"threads_started":1}
I20260928 16:40:08.020851 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.978110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32942 (local address 127.10.247.194:41829)
0928 16:40:08.006059 (+ 27949us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:08.006072 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:08.006115 (+ 43us) server_negotiation.cc:407] Connection header received
0928 16:40:08.006262 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:08.006274 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:08.006691 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:08.006976 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:08.009633 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.011602 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.014986 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.016992 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.017247 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.017491 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:08.020151 (+ 2660us) server_negotiation.cc:299] Negotiation successful
0928 16:40:08.020418 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27662,"thread_start_us":31040,"threads_started":1}
I20260928 16:40:08.025552 24696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.981670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.0.0.1:53740)
0928 16:40:08.011914 (+ 30244us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.011945 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:08.012129 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:08.013555 (+ 1426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:08.013565 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:08.013597 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:08.013966 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.013978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.016211 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.016218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.018312 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.018328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.024524 (+ 6196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.024534 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.024799 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.024841 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 16:40:08.024944 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 16:40:08.025147 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30075,"thread_start_us":36768,"threads_started":1}
I20260928 16:40:08.026748 24468 tablet_service.cc:1467] Processing CreateTablet for tablet 2a9eb0a08623486290d39eb3ff71b7d0 (DEFAULT_TABLE table=test-table [id=8d0f7ac8e4934d3f87c9fab702414253]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:08.028990 24468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a9eb0a08623486290d39eb3ff71b7d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:08.029042 24697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.982470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.10.247.193:37469)
0928 16:40:08.011139 (+ 28669us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:08.011150 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:08.012328 (+ 1178us) server_negotiation.cc:407] Connection header received
0928 16:40:08.012482 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:08.012493 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:08.012923 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:08.013212 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:08.014271 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.015979 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.019854 (+ 3875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.023773 (+ 3919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.024039 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.024271 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:08.026358 (+ 2087us) server_negotiation.cc:299] Negotiation successful
0928 16:40:08.026598 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28417,"thread_start_us":28669,"threads_started":1}
I20260928 16:40:08.034298 24324 tablet_service.cc:1467] Processing CreateTablet for tablet 2a9eb0a08623486290d39eb3ff71b7d0 (DEFAULT_TABLE table=test-table [id=8d0f7ac8e4934d3f87c9fab702414253]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:08.036413 24324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a9eb0a08623486290d39eb3ff71b7d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:08.049567 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.977546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:37563 (local address 127.0.0.1:34602)
0928 16:40:08.008885 (+ 31339us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.008914 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:08.009065 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:08.032521 (+ 23456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:08.032528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:08.032576 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:08.033007 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.033024 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.043903 (+ 10879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.043912 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.046169 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.046195 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.048500 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.048529 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.048775 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.048824 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 16:40:08.048908 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 16:40:08.049096 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31161,"mutex_wait_us":795,"thread_start_us":25071,"threads_started":1}
I20260928 16:40:08.051224 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:07.977605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34602 (local address 127.10.247.195:37563)
0928 16:40:08.031722 (+ 54117us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:08.031733 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:08.031775 (+ 42us) server_negotiation.cc:407] Connection header received
0928 16:40:08.031902 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:08.031911 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:08.032225 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:08.032493 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:08.033267 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.043631 (+ 10364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.046544 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.048209 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.050151 (+ 1942us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.050411 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:08.050507 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 16:40:08.050725 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53866,"thread_start_us":50892,"threads_started":1}
I20260928 16:40:08.057296 24604 tablet_service.cc:1467] Processing CreateTablet for tablet 2a9eb0a08623486290d39eb3ff71b7d0 (DEFAULT_TABLE table=test-table [id=8d0f7ac8e4934d3f87c9fab702414253]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:08.059388 24604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a9eb0a08623486290d39eb3ff71b7d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:08.073017 24698 tablet_bootstrap.cc:492] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: Bootstrap starting.
I20260928 16:40:08.080142 24698 tablet_bootstrap.cc:654] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:08.082120 24698 log.cc:824] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:08.083856 24699 tablet_bootstrap.cc:492] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: Bootstrap starting.
I20260928 16:40:08.085645 24700 tablet_bootstrap.cc:492] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: Bootstrap starting.
I20260928 16:40:08.088330 24699 tablet_bootstrap.cc:654] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:08.090049 24699 log.cc:824] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:08.092083 24700 tablet_bootstrap.cc:654] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:08.093701 24700 log.cc:824] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:08.094105 24698 tablet_bootstrap.cc:492] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: No bootstrap required, opened a new log
I20260928 16:40:08.094499 24698 ts_tablet_manager.cc:1397] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.010s
I20260928 16:40:08.094722 24699 tablet_bootstrap.cc:492] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: No bootstrap required, opened a new log
I20260928 16:40:08.095095 24699 ts_tablet_manager.cc:1397] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260928 16:40:08.105487 24700 tablet_bootstrap.cc:492] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: No bootstrap required, opened a new log
I20260928 16:40:08.105922 24700 ts_tablet_manager.cc:1397] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.007s
I20260928 16:40:08.106379 24699 raft_consensus.cc:348] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } }
I20260928 16:40:08.107155 24699 raft_consensus.cc:374] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:08.107314 24699 raft_consensus.cc:729] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49aed82563d94280afb3e82c012ede18, State: Initialized, Role: FOLLOWER
I20260928 16:40:08.108352 24699 consensus_queue.cc:260] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } }
I20260928 16:40:08.109020 24698 raft_consensus.cc:348] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } }
I20260928 16:40:08.109545 24698 raft_consensus.cc:374] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:08.109651 24698 raft_consensus.cc:729] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80bd7b07d184718aa55c3d21c607ad2, State: Initialized, Role: FOLLOWER
I20260928 16:40:08.110348 24698 consensus_queue.cc:260] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } }
I20260928 16:40:08.118445 24700 raft_consensus.cc:348] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } }
I20260928 16:40:08.119263 24700 raft_consensus.cc:374] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:08.119390 24700 raft_consensus.cc:729] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 740607d98bce48d8a8de456a31373498, State: Initialized, Role: FOLLOWER
I20260928 16:40:08.121328 24700 consensus_queue.cc:260] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } }
I20260928 16:40:08.122655 24698 ts_tablet_manager.cc:1428] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: Time spent starting tablet: real 0.028s user 0.013s sys 0.012s
I20260928 16:40:08.129581 24389 heartbeater.cc:503] Master 127.10.247.254:37513 was elected leader, sending a full tablet report...
I20260928 16:40:08.130681 24699 ts_tablet_manager.cc:1428] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: Time spent starting tablet: real 0.035s user 0.017s sys 0.016s
I20260928 16:40:08.132292 24700 ts_tablet_manager.cc:1428] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: Time spent starting tablet: real 0.026s user 0.008s sys 0.010s
I20260928 16:40:08.132808 24669 heartbeater.cc:503] Master 127.10.247.254:37513 was elected leader, sending a full tablet report...
I20260928 16:40:08.135071 24706 raft_consensus.cc:482] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:40:08.135399 24706 raft_consensus.cc:504] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } }
I20260928 16:40:08.140044 24706 leader_election.cc:288] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f80bd7b07d184718aa55c3d21c607ad2 (127.10.247.194:41829), 49aed82563d94280afb3e82c012ede18 (127.10.247.193:37469)
I20260928 16:40:08.145694 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.137888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:41829 (local address 127.10.247.195:36159)
0928 16:40:08.138340 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.138363 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:08.138502 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:08.139128 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:08.139133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:08.139160 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:08.139629 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.139645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.141181 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.141189 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.143076 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.143093 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.144924 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.144928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.145083 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.145110 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 16:40:08.145192 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 16:40:08.145344 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313}
I20260928 16:40:08.145687 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.138427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:36159 (local address 127.10.247.194:41829)
0928 16:40:08.138637 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:08.138643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:08.138663 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:40:08.138764 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:08.138769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:08.138850 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:08.138937 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:08.139813 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.141034 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.143278 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.144579 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.144744 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.144818 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:08.145249 (+ 431us) server_negotiation.cc:299] Negotiation successful
0928 16:40:08.145396 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 16:40:08.147053 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.138130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.10.247.195:39923)
0928 16:40:08.140093 (+ 1963us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.140110 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:08.140291 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:08.140938 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:08.140944 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:08.140970 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:08.141385 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.141395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.142465 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.142469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.144322 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:08.144338 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.146187 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.146193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:08.146335 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.146362 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 16:40:08.146438 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 16:40:08.146575 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1834,"mutex_wait_us":1,"thread_start_us":1610,"threads_started":1}
I20260928 16:40:08.147046 24697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.138765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:39923 (local address 127.10.247.193:37469)
0928 16:40:08.138997 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:08.139005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:08.140192 (+ 1187us) server_negotiation.cc:407] Connection header received
0928 16:40:08.140494 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:08.140501 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:08.140596 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:08.140770 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:08.141553 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.142309 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.144548 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.145947 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.146139 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.146530 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:08.146594 (+ 64us) server_negotiation.cc:299] Negotiation successful
0928 16:40:08.146747 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:40:08.147475 24488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a9eb0a08623486290d39eb3ff71b7d0" candidate_uuid: "740607d98bce48d8a8de456a31373498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80bd7b07d184718aa55c3d21c607ad2" is_pre_election: true
I20260928 16:40:08.148234 24488 raft_consensus.cc:2454] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 740607d98bce48d8a8de456a31373498 in term 0.
I20260928 16:40:08.148205 24344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a9eb0a08623486290d39eb3ff71b7d0" candidate_uuid: "740607d98bce48d8a8de456a31373498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49aed82563d94280afb3e82c012ede18" is_pre_election: true
I20260928 16:40:08.148927 24344 raft_consensus.cc:2454] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 740607d98bce48d8a8de456a31373498 in term 0.
I20260928 16:40:08.149571 24557 leader_election.cc:302] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 740607d98bce48d8a8de456a31373498, f80bd7b07d184718aa55c3d21c607ad2; no voters:
I20260928 16:40:08.150092 24706 raft_consensus.cc:2781] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:40:08.150242 24706 raft_consensus.cc:482] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:40:08.150339 24706 raft_consensus.cc:3037] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:08.152931 24706 raft_consensus.cc:504] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } }
I20260928 16:40:08.153631 24706 leader_election.cc:288] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [CANDIDATE]: Term 1 election: Requested vote from peers f80bd7b07d184718aa55c3d21c607ad2 (127.10.247.194:41829), 49aed82563d94280afb3e82c012ede18 (127.10.247.193:37469)
I20260928 16:40:08.154243 24344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a9eb0a08623486290d39eb3ff71b7d0" candidate_uuid: "740607d98bce48d8a8de456a31373498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49aed82563d94280afb3e82c012ede18"
I20260928 16:40:08.154242 24488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a9eb0a08623486290d39eb3ff71b7d0" candidate_uuid: "740607d98bce48d8a8de456a31373498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80bd7b07d184718aa55c3d21c607ad2"
I20260928 16:40:08.154493 24344 raft_consensus.cc:3037] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:08.154493 24488 raft_consensus.cc:3037] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:08.158151 24344 raft_consensus.cc:2454] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 740607d98bce48d8a8de456a31373498 in term 1.
I20260928 16:40:08.158152 24488 raft_consensus.cc:2454] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 740607d98bce48d8a8de456a31373498 in term 1.
I20260928 16:40:08.158722 24556 leader_election.cc:302] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49aed82563d94280afb3e82c012ede18, 740607d98bce48d8a8de456a31373498; no voters:
I20260928 16:40:08.159128 24706 raft_consensus.cc:2781] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:40:08.159313 24706 raft_consensus.cc:686] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 1 LEADER]: Becoming Leader. State: Replica: 740607d98bce48d8a8de456a31373498, State: Running, Role: LEADER
I20260928 16:40:08.159894 24706 consensus_queue.cc:237] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } }
I20260928 16:40:08.166191 24190 catalog_manager.cc:5495] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 reported cstate change: term changed from 0 to 1, leader changed from <none> to 740607d98bce48d8a8de456a31373498 (127.10.247.195). New cstate: current_term: 1 leader_uuid: "740607d98bce48d8a8de456a31373498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "740607d98bce48d8a8de456a31373498" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 37563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 } health_report { overall_health: UNKNOWN } } }
I20260928 16:40:08.197505 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.190348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.10.247.195:37563)
0928 16:40:08.190584 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:08.190591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:08.190734 (+ 143us) server_negotiation.cc:407] Connection header received
0928 16:40:08.191041 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:08.191054 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:08.191156 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:08.191329 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:40:08.191791 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.192692 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.194607 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.195078 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.195204 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.195994 (+ 790us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:40:08.196795 (+ 801us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:40:08.196965 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:08.197063 (+ 98us) server_negotiation.cc:299] Negotiation successful
0928 16:40:08.197176 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 16:40:08.211959 24344 raft_consensus.cc:1261] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [term 1 FOLLOWER]: Refusing update from remote peer 740607d98bce48d8a8de456a31373498: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:08.212013 24488 raft_consensus.cc:1261] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [term 1 FOLLOWER]: Refusing update from remote peer 740607d98bce48d8a8de456a31373498: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:08.213517 24706 consensus_queue.cc:1035] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f80bd7b07d184718aa55c3d21c607ad2" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:40:08.214078 24707 consensus_queue.cc:1035] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49aed82563d94280afb3e82c012ede18" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:40:08.240981 24713 mvcc.cc:205] Tried to move back new op lower bound from 7334353339220504576 to 7334353339027976192. Current Snapshot: MvccSnapshot[applied={T|T < 7334353339220504576}]
I20260928 16:40:08.245437 24711 mvcc.cc:205] Tried to move back new op lower bound from 7334353339220504576 to 7334353339027976192. Current Snapshot: MvccSnapshot[applied={T|T < 7334353339220504576 or (T in {7334353339220504576})}]
I20260928 16:40:08.261617 24712 mvcc.cc:205] Tried to move back new op lower bound from 7334353339220504576 to 7334353339027976192. Current Snapshot: MvccSnapshot[applied={T|T < 7334353339220504576 or (T in {7334353339220504576})}]
I20260928 16:40:08.264454 24192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:40:08.264894 24192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:40:08.271998 24192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e3641074a5c4601b1f123ecd6701a71: Sending DeleteTablet for 3 replicas of tablet 2a9eb0a08623486290d39eb3ff71b7d0
I20260928 16:40:08.273869 24604 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9eb0a08623486290d39eb3ff71b7d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:08 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:34602
I20260928 16:40:08.274657 24468 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9eb0a08623486290d39eb3ff71b7d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:08 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:32942
I20260928 16:40:08.276711 24324 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9eb0a08623486290d39eb3ff71b7d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:08 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:53740
I20260928 16:40:08.281345 24721 tablet_replica.cc:330] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: stopping tablet replica
I20260928 16:40:08.282061 24721 raft_consensus.cc:2229] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:40:08.282215 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 24229
I20260928 16:40:08.282799 24721 raft_consensus.cc:2258] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:08.289368 24722 tablet_replica.cc:330] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: stopping tablet replica
I20260928 16:40:08.290123 24722 raft_consensus.cc:2229] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:40:08.290938 24722 raft_consensus.cc:2258] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:08.292470 24720 tablet_replica.cc:330] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: stopping tablet replica
I20260928 16:40:08.292995 24720 raft_consensus.cc:2229] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:40:08.293742 24720 raft_consensus.cc:2258] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:08.295749 24721 ts_tablet_manager.cc:1905] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:08.303354 24722 ts_tablet_manager.cc:1905] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:08.305281 24720 ts_tablet_manager.cc:1905] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:08.311168 24697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.283432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.10.247.193:37469)
0928 16:40:08.283664 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:08.283671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:08.283879 (+ 208us) server_negotiation.cc:407] Connection header received
0928 16:40:08.284100 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:08.284106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:08.284227 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:08.284357 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:08.289817 (+ 5460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.291203 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.296083 (+ 4880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.296835 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.297006 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.301166 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:08.301189 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:08.301214 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:08.301291 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:08.306462 (+ 5171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:08.307657 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:08.307671 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:08.307675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:08.307904 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:08.308488 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:08.308493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:08.308497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:08.309196 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:08.309625 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:08.310337 (+ 712us) server_negotiation.cc:299] Negotiation successful
0928 16:40:08.310645 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 16:40:08.311882 24722 ts_tablet_manager.cc:1918] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:08.312098 24722 log.cc:1196] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/2a9eb0a08623486290d39eb3ff71b7d0
I20260928 16:40:08.313030 24384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:08.318725 24721 ts_tablet_manager.cc:1918] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:08.318970 24721 log.cc:1196] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/2a9eb0a08623486290d39eb3ff71b7d0
I20260928 16:40:08.319181 24720 ts_tablet_manager.cc:1918] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:08.319382 24720 log.cc:1196] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/2a9eb0a08623486290d39eb3ff71b7d0
I20260928 16:40:08.319706 24721 ts_tablet_manager.cc:1939] T 2a9eb0a08623486290d39eb3ff71b7d0 P f80bd7b07d184718aa55c3d21c607ad2: Deleting consensus metadata
I20260928 16:40:08.319970 24720 ts_tablet_manager.cc:1939] T 2a9eb0a08623486290d39eb3ff71b7d0 P 740607d98bce48d8a8de456a31373498: Deleting consensus metadata
I20260928 16:40:08.322219 24178 catalog_manager.cc:4841] TS f80bd7b07d184718aa55c3d21c607ad2 (127.10.247.194:41829): tablet 2a9eb0a08623486290d39eb3ff71b7d0 (table test-table [id=8d0f7ac8e4934d3f87c9fab702414253]) successfully deleted
I20260928 16:40:08.322373 24179 catalog_manager.cc:4841] TS 740607d98bce48d8a8de456a31373498 (127.10.247.195:37563): tablet 2a9eb0a08623486290d39eb3ff71b7d0 (table test-table [id=8d0f7ac8e4934d3f87c9fab702414253]) successfully deleted
I20260928 16:40:08.417399 24722 ts_tablet_manager.cc:1939] T 2a9eb0a08623486290d39eb3ff71b7d0 P 49aed82563d94280afb3e82c012ede18: Deleting consensus metadata
I20260928 16:40:08.418211 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 24229
W20260928 16:40:08.441344 24177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.247.193:37469 (error 108)
I20260928 16:40:08.441730 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 24393
I20260928 16:40:08.443934 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.443260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.0.0.1:53760)
0928 16:40:08.443481 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.443657 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260928 16:40:08.444305 24177 catalog_manager.cc:4584] TS 49aed82563d94280afb3e82c012ede18 (127.10.247.193:37469): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a9eb0a08623486290d39eb3ff71b7d0: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37469: connect: Connection refused (error 111)
I20260928 16:40:08.458743 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.443366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32948 (local address 127.10.247.194:41829)
0928 16:40:08.443570 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:08.443576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:08.443598 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:40:08.443683 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:08.443689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:08.443790 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:08.443917 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:08.445850 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.446677 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.448125 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.448587 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.448886 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.451954 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:08.451971 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:08.451990 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:08.452042 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:08.455363 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:08.456208 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:08.456217 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:08.456220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:08.456378 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:08.456975 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:08.456981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:08.456985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:08.457437 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:08.457777 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:08.458096 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 16:40:08.458356 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260928 16:40:08.459619 24528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:08.499053 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.498334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.0.0.1:53772)
0928 16:40:08.498651 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.498808 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260928 16:40:08.500465 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.499996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.0.0.1:53786)
0928 16:40:08.500164 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.500278 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 16:40:08.557407 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 24393
I20260928 16:40:08.561376 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.560739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.0.0.1:53796)
0928 16:40:08.561007 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.561140 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260928 16:40:08.562788 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.562336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.0.0.1:53800)
0928 16:40:08.562516 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.562625 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 16:40:08.574265 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 24538
I20260928 16:40:08.590245 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.575196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.10.247.195:37563)
0928 16:40:08.575417 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:08.575424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:08.575471 (+ 47us) server_negotiation.cc:407] Connection header received
0928 16:40:08.575753 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:08.575761 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:08.575894 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:08.576013 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:08.577661 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.578569 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.579931 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:08.580419 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:08.580545 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:08.583275 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:08.583292 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:08.583310 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:08.583364 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:08.586991 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:08.587805 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:08.587815 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:08.587817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:08.587990 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:08.588398 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:08.588402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:08.588405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:08.588898 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:08.589276 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:08.589585 (+ 309us) server_negotiation.cc:299] Negotiation successful
0928 16:40:08.589838 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260928 16:40:08.591125 24664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:08.643919 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.643149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.0.0.1:53816)
0928 16:40:08.643465 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.643623 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260928 16:40:08.645344 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.644837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.0.0.1:53826)
0928 16:40:08.645077 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.645181 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260928 16:40:08.731010 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 24538
I20260928 16:40:08.749423 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 24159
I20260928 16:40:08.751122 24222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:08.882903 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:08.882305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37469 (local address 127.0.0.1:53834)
0928 16:40:08.882537 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:08.882661 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260928 16:40:08.882941 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 24159
I20260928 16:40:08.905337 11231 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 16:40:09.242245 11231 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 16:40:09.242472 11231 mini_ranger.cc:66] Stopping Ranger...
I20260928 16:40:09.589105 11231 mini_ranger.cc:68] Stopped Ranger
2026-09-28 16:40:09.589 UTC [23206] LOG: received smart shutdown request
2026-09-28 16:40:09.592 UTC [23206] LOG: background worker "logical replication launcher" (PID 23215) exited with exit code 1
2026-09-28 16:40:09.593 UTC [23210] LOG: shutting down
2026-09-28 16:40:09.760 UTC [23206] LOG: database system is shut down
2026-09-28T16:40:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (60103 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:40:10 dist-test-slave-9fn6 krb5kdc[24736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:40:10 dist-test-slave-9fn6 krb5kdc[24736](info): set up 2 sockets
Sep 28 16:40:10 dist-test-slave-9fn6 krb5kdc[24736](info): commencing operation
krb5kdc: starting...
W20260928 16:40:10.942443 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:40:11 dist-test-slave-9fn6 krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:40:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:40:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:40:11.238371 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37789
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:37665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37789,127.10.247.253:34663,127.10.247.252:45605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:11.639947 24749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:11.640364 24749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:11.640501 24749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:11.648730 24749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:11.648885 24749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:11.648942 24749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:11.649027 24749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:11.649099 24749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:11.660583 24749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:37665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37789,127.10.247.253:34663,127.10.247.252:45605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:11.662555 24749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:11.664240 24749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:11.678133 24755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:11.678084 24757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:11.678227 24754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:11.678972 24749 server_base.cc:1034] running on GCE node
I20260928 16:40:11.679919 24749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:11.681744 24749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:11.682960 24749 hybrid_clock.cc:648] HybridClock initialized: now 1790613611682909 us; error 47 us; skew 500 ppm
I20260928 16:40:11.683377 24749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:11 dist-test-slave-9fn6 krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613611, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:11.687034 24749 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:40:11.688869 24749 webserver.cc:460] Webserver started at http://127.10.247.254:41637/ using document root <none> and password file <none>
I20260928 16:40:11.689723 24749 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:11.689829 24749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:11.690135 24749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:11.693929 24749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "2f9f99e621b2448fa56279ce59aa7696"
format_stamp: "Formatted at 2026-09-28 16:40:11 on dist-test-slave-9fn6"
server_key: "aa6f1e3c5291d78b3ea9a0bf7b7580e5"
server_key_iv: "da835794ae750589531f389f56941914"
server_key_version: "encryptionkey@0"
I20260928 16:40:11.694934 24749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "2f9f99e621b2448fa56279ce59aa7696"
format_stamp: "Formatted at 2026-09-28 16:40:11 on dist-test-slave-9fn6"
server_key: "aa6f1e3c5291d78b3ea9a0bf7b7580e5"
server_key_iv: "da835794ae750589531f389f56941914"
server_key_version: "encryptionkey@0"
I20260928 16:40:11.702068 24749 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260928 16:40:11.707617 24764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:11.709453 24749 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260928 16:40:11.709664 24749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "2f9f99e621b2448fa56279ce59aa7696"
format_stamp: "Formatted at 2026-09-28 16:40:11 on dist-test-slave-9fn6"
server_key: "aa6f1e3c5291d78b3ea9a0bf7b7580e5"
server_key_iv: "da835794ae750589531f389f56941914"
server_key_version: "encryptionkey@0"
I20260928 16:40:11.709925 24749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:11.736840 24749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:11.742641 24749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:11.742978 24749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:11.764128 24749 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:37789
I20260928 16:40:11.764143 24815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:37789 every 8 connection(s)
I20260928 16:40:11.765856 24749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:40:11.770573 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 24749
I20260928 16:40:11.770635 24816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:11.770817 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:40:11.771593 11231 external_mini_cluster.cc:1428] Setting key 8045341678bbfda114838a95515faacf
I20260928 16:40:11.779064 24816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:40:11 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613611, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.247.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.253@KRBTEST.COM" created.
I20260928 16:40:11.806334 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:11.785141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38986 (local address 127.10.247.254:37789)
0928 16:40:11.785898 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:11.785908 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:11.785988 (+ 80us) server_negotiation.cc:407] Connection header received
0928 16:40:11.787495 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:11.787501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:11.787806 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:11.788023 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:11.789601 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:11.790996 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:11.792487 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:11.793162 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:11.793348 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:11.797465 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:11.797475 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:11.797493 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:11.797530 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:11.800768 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:11.801683 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:11.801692 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:11.801695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:11.801884 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:11.802327 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:11.802331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:11.802333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:11.803050 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:11.803288 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:11.803936 (+ 648us) server_negotiation.cc:299] Negotiation successful
0928 16:40:11.804422 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":532,"thread_start_us":327,"threads_started":1}
I20260928 16:40:11.806370 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:11.783461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37789 (local address 127.0.0.1:38986)
0928 16:40:11.784472 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:11.784521 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:11.787006 (+ 2485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:11.788052 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:11.788102 (+ 50us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:11.788965 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:11.789388 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:11.789408 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:11.791155 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:11.791159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:11.792347 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:11.792360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:11.793402 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:11.793409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:11.793656 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:11.794425 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:11.794451 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:11.797224 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:11.800925 (+ 3701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:11.800991 (+ 66us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:11.801004 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:11.801516 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:11.802008 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:11.802013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:11.802018 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:11.802205 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:11.803213 (+ 1008us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:11.803222 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:11.803484 (+ 262us) client_negotiation.cc:769] Sending connection context
0928 16:40:11.803768 (+ 284us) client_negotiation.cc:240] Negotiation successful
0928 16:40:11.804581 (+ 813us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":207,"thread_start_us":425,"threads_started":1}
I20260928 16:40:11.810873 24816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:40:11.812372 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:11.811844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58076)
0928 16:40:11.812060 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:11.812207 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260928 16:40:11.812826 24767 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
I20260928 16:40:11.815013 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:11.814473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58082)
0928 16:40:11.814695 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:11.814794 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260928 16:40:11.815263 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:34663: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab.
Entry for principal kudu/127.10.247.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab.
I20260928 16:40:11.835968 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.253:34663
--webserver_interface=127.10.247.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab
--principal=kudu/127.10.247.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:37665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37789,127.10.247.253:34663,127.10.247.252:45605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:40:11.864730 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } attempt: 1
I20260928 16:40:11.866335 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:11.865749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58092)
0928 16:40:11.866001 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:11.866156 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260928 16:40:11.867491 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:11.867072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58108)
0928 16:40:11.867237 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:11.867322 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 16:40:11.867707 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:34663: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
I20260928 16:40:11.935969 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } attempt: 2
I20260928 16:40:11.937498 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:11.936893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58110)
0928 16:40:11.937192 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:11.937306 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 16:40:11.938742 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:11.938271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58114)
0928 16:40:11.938459 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:11.938551 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260928 16:40:11.939028 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:34663: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
I20260928 16:40:12.030345 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } attempt: 3
I20260928 16:40:12.031998 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.031411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58126)
0928 16:40:12.031676 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.031823 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260928 16:40:12.033293 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.032848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58132)
0928 16:40:12.033035 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.033121 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 16:40:12.033504 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:34663: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
I20260928 16:40:12.176813 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } attempt: 4
I20260928 16:40:12.178375 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.177669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58148)
0928 16:40:12.178070 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.178194 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214}
I20260928 16:40:12.179613 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.179166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58152)
0928 16:40:12.179348 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.179455 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 16:40:12.179854 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:34663: Network error: Client connection negotiation failed: client connection to 127.10.247.253:34663: connect: Connection refused (error 111)
W20260928 16:40:12.239756 24824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:12.240216 24824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:12.240362 24824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:12.248827 24824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:12.249011 24824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:12.249096 24824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:12.249177 24824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:12.249239 24824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:12.260530 24824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:37665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37789,127.10.247.253:34663,127.10.247.252:45605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab
--principal=kudu/127.10.247.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.253:34663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.247.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:12.262369 24824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:12.264056 24824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:12.277894 24829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:12.277865 24830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:12.277931 24832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:12.278676 24824 server_base.cc:1034] running on GCE node
I20260928 16:40:12.279728 24824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:12.281643 24824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:12.282891 24824 hybrid_clock.cc:648] HybridClock initialized: now 1790613612282832 us; error 46 us; skew 500 ppm
I20260928 16:40:12.283324 24824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:12 dist-test-slave-9fn6 krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:12.287042 24824 init.cc:377] Logged in from keytab as kudu/127.10.247.253@KRBTEST.COM (short username kudu)
I20260928 16:40:12.288813 24824 webserver.cc:460] Webserver started at http://127.10.247.253:36291/ using document root <none> and password file <none>
I20260928 16:40:12.289691 24824 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:12.289808 24824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:12.290161 24824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:12.293988 24824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/data/instance:
uuid: "4a6493d306e74551b162820ed72042f5"
format_stamp: "Formatted at 2026-09-28 16:40:12 on dist-test-slave-9fn6"
server_key: "54dccc535eff02981491a686180b9f0a"
server_key_iv: "b5cf74f5da90e230c0d3268f7c75a2f0"
server_key_version: "encryptionkey@0"
I20260928 16:40:12.294953 24824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/wal/instance:
uuid: "4a6493d306e74551b162820ed72042f5"
format_stamp: "Formatted at 2026-09-28 16:40:12 on dist-test-slave-9fn6"
server_key: "54dccc535eff02981491a686180b9f0a"
server_key_iv: "b5cf74f5da90e230c0d3268f7c75a2f0"
server_key_version: "encryptionkey@0"
I20260928 16:40:12.302018 24824 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260928 16:40:12.306733 24839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:12.308497 24824 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260928 16:40:12.308724 24824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/wal
uuid: "4a6493d306e74551b162820ed72042f5"
format_stamp: "Formatted at 2026-09-28 16:40:12 on dist-test-slave-9fn6"
server_key: "54dccc535eff02981491a686180b9f0a"
server_key_iv: "b5cf74f5da90e230c0d3268f7c75a2f0"
server_key_version: "encryptionkey@0"
I20260928 16:40:12.308967 24824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:12.332542 24824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:12.337635 24824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:12.337950 24824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:12.358783 24824 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.253:34663
I20260928 16:40:12.358804 24890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.253:34663 every 8 connection(s)
I20260928 16:40:12.360453 24824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
I20260928 16:40:12.365165 24891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:12.367540 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 24824
I20260928 16:40:12.367774 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-1/wal/instance
I20260928 16:40:12.368402 11231 external_mini_cluster.cc:1428] Setting key 7ef6e67974d528b23ebb8cac3221b520
I20260928 16:40:12.375609 24891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:40:12 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.247.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.252@KRBTEST.COM" created.
I20260928 16:40:12.403491 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.379920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38994 (local address 127.10.247.254:37789)
0928 16:40:12.382036 (+ 2116us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:12.382043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:12.383001 (+ 958us) server_negotiation.cc:407] Connection header received
0928 16:40:12.385524 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:12.385531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:12.385636 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:12.385770 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:12.387833 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.389070 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.390921 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.391520 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.391651 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:12.395903 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:12.395912 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:12.395917 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:12.395991 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:12.399793 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:12.400794 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:12.400799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:12.400801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:12.400941 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:12.401471 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:12.401475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:12.401479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:12.401800 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:12.401929 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:12.402943 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0928 16:40:12.403159 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1943,"thread_start_us":1764,"threads_started":1}
I20260928 16:40:12.405318 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.380675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37789 (local address 127.0.0.1:38994)
0928 16:40:12.382735 (+ 2060us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.382805 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:12.385132 (+ 2327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:12.386035 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:12.386045 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:12.386756 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:12.387582 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:12.387606 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.389273 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.389277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:12.390745 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:12.390760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.391786 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.391792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:12.392040 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:12.392890 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:12.392919 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:12.395577 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:12.399956 (+ 4379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:12.399966 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:12.399985 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:12.400686 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:12.401069 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:12.401073 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:12.401075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:12.401342 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:12.401913 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:12.401920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:12.402487 (+ 567us) client_negotiation.cc:769] Sending connection context
0928 16:40:12.402849 (+ 362us) client_negotiation.cc:240] Negotiation successful
0928 16:40:12.403490 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1682,"thread_start_us":1432,"threads_started":1}
I20260928 16:40:12.407529 24891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:40:12 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
I20260928 16:40:12.425735 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.408407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58160)
0928 16:40:12.408652 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.408675 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:12.408815 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:12.410100 (+ 1285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:12.410105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:12.410557 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:12.410731 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:12.410741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.412646 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.412654 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:12.413758 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:12.413773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.415081 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.415086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:12.415309 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:12.416059 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:12.416080 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:12.418784 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:12.422896 (+ 4112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:12.422900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:12.422903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:12.423347 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:12.423815 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:12.423819 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:12.423821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:12.424023 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:12.424860 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:12.424865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:12.425021 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 16:40:12.425149 (+ 128us) client_negotiation.cc:240] Negotiation successful
0928 16:40:12.425413 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260928 16:40:12.426045 24896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.408721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58160 (local address 127.10.247.253:34663)
0928 16:40:12.409492 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:12.409500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:12.409543 (+ 43us) server_negotiation.cc:407] Connection header received
0928 16:40:12.409600 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:12.409609 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:12.409905 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:12.410070 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:12.410900 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.412429 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.413897 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.414880 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.415023 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:12.419023 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:12.419039 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:12.419061 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:12.419103 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:12.422741 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:12.423513 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:12.423524 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:12.423528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:12.423698 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:12.424138 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:12.424143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:12.424145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:12.424722 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:12.424957 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:12.425340 (+ 383us) server_negotiation.cc:299] Negotiation successful
0928 16:40:12.425630 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":399,"threads_started":1}
I20260928 16:40:12.428920 24891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:40:12.430143 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.429649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54478)
0928 16:40:12.429844 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.429962 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 16:40:12.430388 24843 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
I20260928 16:40:12.432087 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.431741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54486)
0928 16:40:12.431888 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.431971 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260928 16:40:12.432284 24891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab.
Entry for principal kudu/127.10.247.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab.
I20260928 16:40:12.440297 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.252:45605
--webserver_interface=127.10.247.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab
--principal=kudu/127.10.247.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:37665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37789,127.10.247.253:34663,127.10.247.252:45605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:40:12.479146 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } attempt: 5
I20260928 16:40:12.481642 24891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 1
I20260928 16:40:12.483022 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.482375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54498)
0928 16:40:12.482606 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.482780 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260928 16:40:12.484406 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.483912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54508)
0928 16:40:12.484116 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.484225 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260928 16:40:12.484758 24891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Sep 28 16:40:12 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613611, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
I20260928 16:40:12.496280 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.480036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58174)
0928 16:40:12.480371 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.480408 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:12.480634 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:12.481186 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:12.481192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:12.481778 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:12.482056 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:12.482072 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.483430 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.483438 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:12.484836 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:12.484860 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.485913 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.485919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:12.486119 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:12.486779 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:12.486800 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:12.489229 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:12.493291 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:12.493296 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:12.493299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:12.493728 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:12.494317 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:12.494321 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:12.494324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:12.494483 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:12.495257 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:12.495261 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:12.495430 (+ 169us) client_negotiation.cc:769] Sending connection context
0928 16:40:12.495612 (+ 182us) client_negotiation.cc:240] Negotiation successful
0928 16:40:12.495900 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260928 16:40:12.496358 24896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.480189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58174 (local address 127.10.247.253:34663)
0928 16:40:12.480396 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:12.480401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:12.480495 (+ 94us) server_negotiation.cc:407] Connection header received
0928 16:40:12.480836 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:12.480841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:12.480996 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:12.481108 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:12.482260 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.483263 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.485028 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.485720 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.485854 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:12.489465 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:12.489473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:12.489478 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:12.489527 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:12.493108 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:12.493908 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:12.493916 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:12.493921 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:12.494132 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:12.494606 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:12.494612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:12.494615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:12.495106 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:12.495272 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:12.495776 (+ 504us) server_negotiation.cc:299] Negotiation successful
0928 16:40:12.496041 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260928 16:40:12.497758 24816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:40:12.499042 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.498586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54514)
0928 16:40:12.498775 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.498872 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 16:40:12.500247 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.499826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54516)
0928 16:40:12.500017 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.500122 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260928 16:40:12.500514 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
I20260928 16:40:12.551824 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 1
I20260928 16:40:12.553071 24891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 2
I20260928 16:40:12.553617 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.552906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54520)
0928 16:40:12.553193 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.553350 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260928 16:40:12.554415 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.553779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54536)
0928 16:40:12.554091 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.554212 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260928 16:40:12.554733 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.554324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54544)
0928 16:40:12.554494 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.554589 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260928 16:40:12.554983 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
I20260928 16:40:12.555581 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.555159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54548)
0928 16:40:12.555339 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.555431 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260928 16:40:12.555826 24891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
I20260928 16:40:12.623226 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 2
I20260928 16:40:12.625221 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.624446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54552)
0928 16:40:12.624841 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.624974 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260928 16:40:12.626492 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.626064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54556)
0928 16:40:12.626252 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.626340 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260928 16:40:12.626737 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
I20260928 16:40:12.647074 24891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 3
I20260928 16:40:12.648674 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.648040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54564)
0928 16:40:12.648289 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.648410 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260928 16:40:12.649960 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.649464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54566)
0928 16:40:12.649732 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.649823 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260928 16:40:12.650193 24891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
I20260928 16:40:12.733029 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 3
I20260928 16:40:12.734503 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.733894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54570)
0928 16:40:12.734160 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.734275 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260928 16:40:12.735671 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.735243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54586)
0928 16:40:12.735452 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.735544 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260928 16:40:12.735895 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
I20260928 16:40:12.793437 24891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 4
I20260928 16:40:12.794898 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.794300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54594)
0928 16:40:12.794590 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.794694 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260928 16:40:12.796152 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.795733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54602)
0928 16:40:12.795904 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.796023 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260928 16:40:12.796377 24891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
W20260928 16:40:12.848752 24899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:12.849190 24899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:12.849345 24899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:12.857926 24899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:12.858145 24899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:12.858239 24899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:12.858325 24899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:12.858392 24899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:12.870262 24899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:37665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37789,127.10.247.253:34663,127.10.247.252:45605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab
--principal=kudu/127.10.247.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.252:45605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.247.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:12.872255 24899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:12.874105 24899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:12.889058 24904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:12.889087 24905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:12.889281 24907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:12.889964 24899 server_base.cc:1034] running on GCE node
I20260928 16:40:12.890995 24899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:12.893136 24899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:12.894415 24899 hybrid_clock.cc:648] HybridClock initialized: now 1790613612894350 us; error 58 us; skew 500 ppm
I20260928 16:40:12.894899 24899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:12 dist-test-slave-9fn6 krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:12.899130 24899 init.cc:377] Logged in from keytab as kudu/127.10.247.252@KRBTEST.COM (short username kudu)
I20260928 16:40:12.901260 24899 webserver.cc:460] Webserver started at http://127.10.247.252:34909/ using document root <none> and password file <none>
I20260928 16:40:12.902222 24899 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:12.902366 24899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:12.902743 24899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:12.906857 24899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/data/instance:
uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003"
format_stamp: "Formatted at 2026-09-28 16:40:12 on dist-test-slave-9fn6"
server_key: "b0f4494a39828dd3c596d00eb8f4c46a"
server_key_iv: "7e8faf76c2c4a892927dac78af0cade0"
server_key_version: "encryptionkey@0"
I20260928 16:40:12.907889 24899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/wal/instance:
uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003"
format_stamp: "Formatted at 2026-09-28 16:40:12 on dist-test-slave-9fn6"
server_key: "b0f4494a39828dd3c596d00eb8f4c46a"
server_key_iv: "7e8faf76c2c4a892927dac78af0cade0"
server_key_version: "encryptionkey@0"
I20260928 16:40:12.913139 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 4
I20260928 16:40:12.914567 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.914022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54618)
0928 16:40:12.914246 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.914352 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260928 16:40:12.915125 24899 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260928 16:40:12.915776 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.915344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54622)
0928 16:40:12.915514 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.915601 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 16:40:12.915977 24816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:45605: Network error: Client connection negotiation failed: client connection to 127.10.247.252:45605: connect: Connection refused (error 111)
I20260928 16:40:12.919814 24914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:12.921700 24899 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260928 16:40:12.921909 24899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/wal
uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003"
format_stamp: "Formatted at 2026-09-28 16:40:12 on dist-test-slave-9fn6"
server_key: "b0f4494a39828dd3c596d00eb8f4c46a"
server_key_iv: "7e8faf76c2c4a892927dac78af0cade0"
server_key_version: "encryptionkey@0"
I20260928 16:40:12.922152 24899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:12.945461 24899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:12.950985 24899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:12.951313 24899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:12.971599 24899 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.252:45605
I20260928 16:40:12.971637 24966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.252:45605 every 8 connection(s)
I20260928 16:40:12.973570 24899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
I20260928 16:40:12.978744 24967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:12.982033 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 24899
I20260928 16:40:12.982272 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790613388455587-11231-0/minicluster-data/master-2/wal/instance
I20260928 16:40:12.982815 11231 external_mini_cluster.cc:1428] Setting key 9ade636013a8a7f9efbcfa2492deee40
I20260928 16:40:12.988842 24967 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:40:12 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
Sep 28 16:40:12 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:13.010438 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.992545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39006 (local address 127.10.247.254:37789)
0928 16:40:12.993222 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:12.993229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:12.993851 (+ 622us) server_negotiation.cc:407] Connection header received
0928 16:40:12.994065 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:12.994070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:12.994165 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:12.994267 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:12.995691 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.996754 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.998293 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.998746 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.998875 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.003109 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.003126 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.003130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.003192 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.006526 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.007555 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.007560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.007563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.007724 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.008183 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.008187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.008189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.008558 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.008796 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.009728 (+ 932us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.010018 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":342,"threads_started":1}
I20260928 16:40:13.011874 24970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.985536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54628 (local address 127.10.247.252:45605)
0928 16:40:12.986561 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:12.986581 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:12.986655 (+ 74us) server_negotiation.cc:407] Connection header received
0928 16:40:12.989172 (+ 2517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:12.989210 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:12.989943 (+ 733us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:12.990511 (+ 568us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:12.992211 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.994474 (+ 2263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.996352 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.997402 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.997606 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.002782 (+ 5176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.002793 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.002811 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.002851 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.006804 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.007846 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.007857 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.007861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.008053 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.008468 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.008474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.008477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.009241 (+ 764us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.009510 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.009951 (+ 441us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.010445 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":202,"server-negotiator.queue_time_us":714,"thread_start_us":446,"threads_started":1}
I20260928 16:40:13.011911 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:12.992541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37789 (local address 127.0.0.1:39006)
0928 16:40:12.993672 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:12.993727 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:12.993936 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:12.994366 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:12.994374 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:12.995199 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:12.995497 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:12.995507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.996933 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.996937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:12.998133 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:12.998147 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:12.999010 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:12.999018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:12.999189 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:12.999846 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:12.999872 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:13.002858 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:13.006719 (+ 3861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.006732 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.006746 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.007392 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.007844 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.007848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.007850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.008062 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.008748 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:13.008756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:13.009259 (+ 503us) client_negotiation.cc:769] Sending connection context
0928 16:40:13.009625 (+ 366us) client_negotiation.cc:240] Negotiation successful
0928 16:40:13.010292 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":32,"thread_start_us":625,"threads_started":1}
I20260928 16:40:13.013990 24967 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:40:13 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
Sep 28 16:40:13 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
I20260928 16:40:13.032810 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.015974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.10.247.254:37789)
0928 16:40:13.016196 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:13.016202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:13.016219 (+ 17us) server_negotiation.cc:407] Connection header received
0928 16:40:13.016278 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:13.016282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:13.016362 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:13.016451 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:13.017969 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.018767 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.020232 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.020780 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.020910 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.025913 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.025920 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.025923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.025973 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.029616 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.030551 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.030557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.030561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.030732 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.031095 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.031101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.031105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.031516 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.031854 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.032099 (+ 245us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.032371 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:40:13.037288 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.015049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:34663 (local address 127.0.0.1:58182)
0928 16:40:13.015260 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:13.015284 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:13.015405 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:13.022420 (+ 7015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:13.022427 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:13.022877 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:13.023073 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:13.023084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.024370 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.024375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:13.025478 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:13.025496 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.026391 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.026398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:13.026677 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.027344 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:13.027371 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:13.030066 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:13.034244 (+ 4178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.034250 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.034253 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.034793 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.035330 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.035335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.035338 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.035558 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.036155 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:13.036161 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:13.036347 (+ 186us) client_negotiation.cc:769] Sending connection context
0928 16:40:13.036494 (+ 147us) client_negotiation.cc:240] Negotiation successful
0928 16:40:13.036896 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260928 16:40:13.037379 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.015013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.10.247.253:34663)
0928 16:40:13.021987 (+ 6974us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:13.021993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:13.022013 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:40:13.022078 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:13.022082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:13.022161 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:13.022284 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:13.023272 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.024170 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.025667 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.026196 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.026317 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.030290 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.030295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.030299 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.030341 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.034086 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.034963 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.034969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.034972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.035202 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.035707 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.035710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.035713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.036026 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.036178 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.036644 (+ 466us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.036897 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6827,"thread_start_us":6657,"threads_started":1}
I20260928 16:40:13.038511 24967 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:40:13 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
Sep 28 16:40:13 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
I20260928 16:40:13.052891 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.039346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54644)
0928 16:40:13.039518 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:13.039538 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:13.039688 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:13.039977 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:13.039980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:13.040393 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:13.040543 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:13.040552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.041654 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.041659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:13.042546 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:13.042557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.043256 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.043261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:13.043444 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.044105 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:13.044133 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:13.046503 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:13.050139 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.050144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.050148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.050588 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.050955 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.050961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.050964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.051116 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.051758 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:13.051763 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:13.051966 (+ 203us) client_negotiation.cc:769] Sending connection context
0928 16:40:13.052118 (+ 152us) client_negotiation.cc:240] Negotiation successful
0928 16:40:13.052501 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260928 16:40:13.053093 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.034829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.10.247.253:34663)
0928 16:40:13.035586 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:13.035592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:13.035616 (+ 24us) server_negotiation.cc:407] Connection header received
0928 16:40:13.035680 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:13.035685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:13.035760 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:13.035853 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:13.037382 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.038547 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.040160 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.040687 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.040819 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.045952 (+ 5133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.045961 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.045965 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.046011 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.049445 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.050365 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.050370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.050373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.050524 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.050975 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.050980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.050982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.051422 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.051763 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.052084 (+ 321us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.052378 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":308,"threads_started":1}
I20260928 16:40:13.053188 24970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.039364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54644 (local address 127.10.247.252:45605)
0928 16:40:13.039545 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:13.039551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:13.039623 (+ 72us) server_negotiation.cc:407] Connection header received
0928 16:40:13.039780 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:13.039785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:13.039864 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:13.039970 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:13.040700 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.041512 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.042686 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.043121 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.043231 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.046705 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.046712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.046717 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.046760 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.049992 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.050707 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.050712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.050714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.050843 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.051231 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.051236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.051239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.051612 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.051761 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.052243 (+ 482us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.052744 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":71}
I20260928 16:40:13.059440 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:40:13.060027 24967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003: Bootstrap starting.
I20260928 16:40:13.065276 24967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:13.066975 24967 log.cc:824] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:13.086349 24967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003: No bootstrap required, opened a new log
I20260928 16:40:13.096493 24891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 5
I20260928 16:40:13.099205 24967 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.099823 24967 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:13.099936 24967 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc56ebb4cee14f1b86f2f1f8c1fdc003, State: Initialized, Role: FOLLOWER
I20260928 16:40:13.100426 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.067094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58198 (local address 127.10.247.253:34663)
0928 16:40:13.067367 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:13.067375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:13.079121 (+ 11746us) server_negotiation.cc:407] Connection header received
0928 16:40:13.079704 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:13.079712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:13.079819 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:13.079901 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:13.083083 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.084275 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.086544 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.087235 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.087352 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.092204 (+ 4852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.092215 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.092221 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.092282 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.096287 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.097658 (+ 1371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.097666 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.097670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.097853 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.098334 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.098340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.098343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.098776 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.099057 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.099664 (+ 607us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.099955 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260928 16:40:13.100551 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.066504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.10.247.254:37789)
0928 16:40:13.066766 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:13.066774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:13.081695 (+ 14921us) server_negotiation.cc:407] Connection header received
0928 16:40:13.081806 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:13.081812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:13.081923 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:13.082025 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:13.084384 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.085602 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.087814 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.088492 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.088630 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.092440 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.092447 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.092451 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.092502 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.096733 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.097909 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.097917 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.097920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.098111 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.098585 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.098591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.098593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.098997 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.099237 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.099902 (+ 665us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.100167 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260928 16:40:13.100836 24967 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.105305 24970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.066116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54660 (local address 127.10.247.252:45605)
0928 16:40:13.066768 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:13.066774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:13.079321 (+ 12547us) server_negotiation.cc:407] Connection header received
0928 16:40:13.079704 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:13.079712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:13.079819 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:13.079901 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:13.083357 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.084515 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.086620 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.087426 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.087534 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.092250 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.092258 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.092263 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.092310 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.096292 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.102246 (+ 5954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.102257 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.102262 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.102455 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.102934 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.102940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.102943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.103444 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.103751 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.104497 (+ 746us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.104917 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515}
I20260928 16:40:13.105383 24984 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.105937 24984 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:40:13.107285 24967 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [sys.catalog]: configured and running, proceeding with master startup.
Sep 28 16:40:13 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
I20260928 16:40:13.118737 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.097528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54672)
0928 16:40:13.097995 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:13.098029 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:13.098138 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:13.102657 (+ 4519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:13.102662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:13.103228 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:13.103487 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:13.103501 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.105654 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.105662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:13.106915 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:13.106930 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.107764 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.107770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:13.108072 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.108750 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:13.108776 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:13.111411 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:13.115628 (+ 4217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.115633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.115637 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.116169 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.116677 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.116693 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.116698 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.116875 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.117608 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:13.117613 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:13.117802 (+ 189us) client_negotiation.cc:769] Sending connection context
0928 16:40:13.117953 (+ 151us) client_negotiation.cc:240] Negotiation successful
0928 16:40:13.118331 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311}
I20260928 16:40:13.119973 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.098369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.10.247.252:45605)
0928 16:40:13.102276 (+ 3907us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:13.102282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:13.102308 (+ 26us) server_negotiation.cc:407] Connection header received
0928 16:40:13.102365 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:13.102369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:13.102453 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:13.103659 (+ 1206us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:13.104160 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.105453 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.107080 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.107608 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.107722 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.111700 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.111708 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.111714 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.111765 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.115466 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.116298 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.116303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.116307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.116531 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.116985 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.116991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.116994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.117435 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.119023 (+ 1588us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.119166 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.119453 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3729,"thread_start_us":2811,"threads_started":1}
W20260928 16:40:13.120756 24995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:40:13.120893 24995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:40:13.126102 24891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5: Bootstrap starting.
I20260928 16:40:13.117586 24899 master_runner.cc:186] Error getting master registration for 127.10.247.254:37789: OK, instance_id { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" instance_seqno: 1790613611758100 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:40:13.128394 24899 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:40:13.130820 24891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:13.132251 24891 log.cc:824] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:13.135869 24891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5: No bootstrap required, opened a new log
I20260928 16:40:13.145080 24891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.145522 24891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:13.145642 24891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a6493d306e74551b162820ed72042f5, State: Initialized, Role: FOLLOWER
I20260928 16:40:13.146382 24891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.147478 24998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.147974 24998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:40:13.149183 24891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 16:40:13.167191 25009 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:40:13.167515 25009 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:40:13.157498 24824 master_runner.cc:186] Error getting master registration for 127.10.247.254:37789: OK, instance_id { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" instance_seqno: 1790613611758100 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:40:13.169433 24824 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:40:13.193219 24816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } attempt: 5
W20260928 16:40:13.204077 24920 tablet.cc:2369] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 16:40:13 dist-test-slave-9fn6 krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613611, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
I20260928 16:40:13.215757 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.194257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:45605 (local address 127.0.0.1:54674)
0928 16:40:13.194825 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:13.194851 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:13.195000 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:13.196766 (+ 1766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:13.196773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:13.197404 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:13.197692 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:13.197707 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.199344 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.199351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:13.201012 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:13.201033 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.202217 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.202225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:13.202438 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.203293 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:13.203322 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:13.207031 (+ 3709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:13.212024 (+ 4993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.212030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.212035 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.212705 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.213300 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:13.213306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:13.213310 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:13.213574 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:13.214371 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:13.214377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:13.214616 (+ 239us) client_negotiation.cc:769] Sending connection context
0928 16:40:13.214824 (+ 208us) client_negotiation.cc:240] Negotiation successful
0928 16:40:13.215222 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386}
I20260928 16:40:13.215835 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:13.195282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.10.247.252:45605)
0928 16:40:13.195784 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:13.195791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:13.195812 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:40:13.195875 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:13.195880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:13.195992 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:13.196102 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:13.197893 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.199170 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.201237 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:13.202020 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:13.202176 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:13.207323 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:13.207332 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:13.207338 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:13.207404 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:13.211816 (+ 4412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.212871 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.212877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.212881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.213073 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:13.213715 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:13.213720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:13.213724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:13.214180 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:13.214475 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:13.214957 (+ 482us) server_negotiation.cc:299] Negotiation successful
0928 16:40:13.215286 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354}
I20260928 16:40:13.223841 24816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696: Bootstrap starting.
I20260928 16:40:13.230247 24816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:13.231984 24816 log.cc:824] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:13.236402 24816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696: No bootstrap required, opened a new log
I20260928 16:40:13.249852 24816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.250434 24816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:13.250592 24816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9f99e621b2448fa56279ce59aa7696, State: Initialized, Role: FOLLOWER
I20260928 16:40:13.251458 24816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.252859 25012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.253465 25012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:40:13.254824 24816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:40:13.295156 24749 master_runner.cc:263] No leader master found from master 2f9f99e621b2448fa56279ce59aa7696
I20260928 16:40:13.295361 24749 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260928 16:40:13.298159 25023 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:40:13.298453 25023 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260928 16:40:13.341406 24845 tablet.cc:2369] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:40:13.360314 25012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:40:13.360715 25012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.362808 25012 leader_election.cc:288] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a6493d306e74551b162820ed72042f5 (127.10.247.253:34663), dc56ebb4cee14f1b86f2f1f8c1fdc003 (127.10.247.252:45605)
I20260928 16:40:13.363955 24998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:40:13.364321 24998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.365751 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2f9f99e621b2448fa56279ce59aa7696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" is_pre_election: true
I20260928 16:40:13.366564 24941 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9f99e621b2448fa56279ce59aa7696 in term 0.
I20260928 16:40:13.368855 24998 leader_election.cc:288] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9f99e621b2448fa56279ce59aa7696 (127.10.247.254:37789), dc56ebb4cee14f1b86f2f1f8c1fdc003 (127.10.247.252:45605)
I20260928 16:40:13.369984 24866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2f9f99e621b2448fa56279ce59aa7696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a6493d306e74551b162820ed72042f5" is_pre_election: true
I20260928 16:40:13.370579 24866 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9f99e621b2448fa56279ce59aa7696 in term 0.
I20260928 16:40:13.371989 24768 leader_election.cc:302] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9f99e621b2448fa56279ce59aa7696, dc56ebb4cee14f1b86f2f1f8c1fdc003; no voters:
I20260928 16:40:13.372929 24791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a6493d306e74551b162820ed72042f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9f99e621b2448fa56279ce59aa7696" is_pre_election: true
I20260928 16:40:13.373546 24791 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a6493d306e74551b162820ed72042f5 in term 0.
I20260928 16:40:13.375201 24841 leader_election.cc:302] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9f99e621b2448fa56279ce59aa7696, 4a6493d306e74551b162820ed72042f5; no voters:
I20260928 16:40:13.375970 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a6493d306e74551b162820ed72042f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" is_pre_election: true
I20260928 16:40:13.376284 24941 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a6493d306e74551b162820ed72042f5 in term 0.
I20260928 16:40:13.381623 24998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:40:13.381834 24998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:40:13.381937 24998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:13.385855 24998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.387668 24791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a6493d306e74551b162820ed72042f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9f99e621b2448fa56279ce59aa7696"
I20260928 16:40:13.387969 24791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:13.392050 24791 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a6493d306e74551b162820ed72042f5 in term 1.
I20260928 16:40:13.392577 25012 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260928 16:40:13.393513 24841 leader_election.cc:302] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9f99e621b2448fa56279ce59aa7696, 4a6493d306e74551b162820ed72042f5; no voters:
I20260928 16:40:13.398473 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a6493d306e74551b162820ed72042f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003"
I20260928 16:40:13.398833 24941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:13.407959 24941 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a6493d306e74551b162820ed72042f5 in term 1.
I20260928 16:40:13.411401 24998 leader_election.cc:288] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9f99e621b2448fa56279ce59aa7696 (127.10.247.254:37789), dc56ebb4cee14f1b86f2f1f8c1fdc003 (127.10.247.252:45605)
I20260928 16:40:13.411998 24998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:40:13.412540 24998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4a6493d306e74551b162820ed72042f5, State: Running, Role: LEADER
I20260928 16:40:13.413838 24998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } }
I20260928 16:40:13.430157 25025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a6493d306e74551b162820ed72042f5. Latest consensus state: current_term: 1 leader_uuid: "4a6493d306e74551b162820ed72042f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.430991 25025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:13.444398 25027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:40:13.447417 25027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:40:13.471685 24941 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [term 1 FOLLOWER]: Refusing update from remote peer 4a6493d306e74551b162820ed72042f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:13.473160 25025 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:40:13.482203 24791 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [term 1 FOLLOWER]: Refusing update from remote peer 4a6493d306e74551b162820ed72042f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:13.483491 24998 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 16:40:13.497129 24770 tablet.cc:2369] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:40:13.528702 25012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a6493d306e74551b162820ed72042f5. Latest consensus state: current_term: 1 leader_uuid: "4a6493d306e74551b162820ed72042f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.529211 25012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:40:13.535368 25028 mvcc.cc:205] Tried to move back new op lower bound from 7334353360744833024 to 7334353360559874048. Current Snapshot: MvccSnapshot[applied={T|T < 7334353360744833024}]
I20260928 16:40:13.550900 25030 mvcc.cc:205] Tried to move back new op lower bound from 7334353360744833024 to 7334353360559874048. Current Snapshot: MvccSnapshot[applied={T|T < 7334353360744833024}]
I20260928 16:40:13.562292 24984 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a6493d306e74551b162820ed72042f5. Latest consensus state: current_term: 1 leader_uuid: "4a6493d306e74551b162820ed72042f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.562832 24984 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:40:13.569784 24998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a6493d306e74551b162820ed72042f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.570315 24998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:13.573136 25012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a6493d306e74551b162820ed72042f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.573551 25012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:40:13.575302 25029 mvcc.cc:205] Tried to move back new op lower bound from 7334353360744833024 to 7334353360559874048. Current Snapshot: MvccSnapshot[applied={T|T < 7334353360744833024}]
I20260928 16:40:13.577610 25027 catalog_manager.cc:1347] Generated new cluster ID: 7fdf565ac5de40339707097851d181c7
I20260928 16:40:13.577759 25027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:40:13.580901 24998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a6493d306e74551b162820ed72042f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.581390 24998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5 [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:13.610209 24984 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a6493d306e74551b162820ed72042f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f99e621b2448fa56279ce59aa7696" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37789 } } peers { permanent_uuid: "4a6493d306e74551b162820ed72042f5" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 34663 } } peers { permanent_uuid: "dc56ebb4cee14f1b86f2f1f8c1fdc003" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 45605 } } }
I20260928 16:40:13.610610 24984 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:40:13.624478 25027 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:40:13.629297 25027 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:40:13.650614 25027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a6493d306e74551b162820ed72042f5: Generated new TSK 0
I20260928 16:40:13.652328 25027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:40:14.123667 24995 catalog_manager.cc:1259] Loaded cluster ID: 7fdf565ac5de40339707097851d181c7
I20260928 16:40:14.123822 24995 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003: loading cluster ID for follower catalog manager: success
I20260928 16:40:14.127089 24995 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003: acquiring CA information for follower catalog manager: success
I20260928 16:40:14.129733 24995 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dc56ebb4cee14f1b86f2f1f8c1fdc003: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:40:14.301316 25023 catalog_manager.cc:1259] Loaded cluster ID: 7fdf565ac5de40339707097851d181c7
I20260928 16:40:14.301447 25023 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696: loading cluster ID for follower catalog manager: success
I20260928 16:40:14.304410 25023 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696: acquiring CA information for follower catalog manager: success
I20260928 16:40:14.306756 25023 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2f9f99e621b2448fa56279ce59aa7696: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:40:14.779987 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 24749
I20260928 16:40:14.781639 24811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:14.905246 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 24749
I20260928 16:40:14.927542 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 24824
I20260928 16:40:14.929196 24886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:15.055752 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 24824
I20260928 16:40:15.077692 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 24899
I20260928 16:40:15.079144 24962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:15.205057 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 24899
2026-09-28T16:40:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5316 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:40:15 dist-test-slave-9fn6 krb5kdc[25045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:40:15 dist-test-slave-9fn6 krb5kdc[25045](info): set up 2 sockets
Sep 28 16:40:15 dist-test-slave-9fn6 krb5kdc[25045](info): commencing operation
krb5kdc: starting...
W20260928 16:40:16.281422 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:40:16 dist-test-slave-9fn6 krb5kdc[25045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:40:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:40:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:40:16.584539 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:41253
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:41253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:16.981666 25058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:16.982107 25058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:16.982254 25058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:16.990530 25058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:16.990715 25058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:16.990803 25058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:16.990887 25058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:16.990950 25058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:17.003111 25058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:41253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:41253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:17.005223 25058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:17.007050 25058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:17.021262 25063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:17.021175 25066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:17.021207 25064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:17.022379 25058 server_base.cc:1034] running on GCE node
I20260928 16:40:17.023473 25058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:17.025508 25058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:17.026741 25058 hybrid_clock.cc:648] HybridClock initialized: now 1790613617026676 us; error 72 us; skew 500 ppm
I20260928 16:40:17.027132 25058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:17 dist-test-slave-9fn6 krb5kdc[25045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613617, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:17.030727 25058 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:40:17.032467 25058 webserver.cc:460] Webserver started at http://127.10.247.254:44017/ using document root <none> and password file <none>
I20260928 16:40:17.033334 25058 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:17.033449 25058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:17.033773 25058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:17.037465 25058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "3cf3512ffad94903b83ccea9581f581a"
format_stamp: "Formatted at 2026-09-28 16:40:17 on dist-test-slave-9fn6"
server_key: "2e9d42c31473642d615c89518e57f47e"
server_key_iv: "0c43b7f7724113cf55ad48fa20fda41f"
server_key_version: "encryptionkey@0"
I20260928 16:40:17.038257 25058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "3cf3512ffad94903b83ccea9581f581a"
format_stamp: "Formatted at 2026-09-28 16:40:17 on dist-test-slave-9fn6"
server_key: "2e9d42c31473642d615c89518e57f47e"
server_key_iv: "0c43b7f7724113cf55ad48fa20fda41f"
server_key_version: "encryptionkey@0"
I20260928 16:40:17.044080 25058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260928 16:40:17.048223 25073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:17.049932 25058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:40:17.050112 25058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "3cf3512ffad94903b83ccea9581f581a"
format_stamp: "Formatted at 2026-09-28 16:40:17 on dist-test-slave-9fn6"
server_key: "2e9d42c31473642d615c89518e57f47e"
server_key_iv: "0c43b7f7724113cf55ad48fa20fda41f"
server_key_version: "encryptionkey@0"
I20260928 16:40:17.050331 25058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:17.069473 25058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:17.074262 25058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:17.074560 25058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:17.094957 25058 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:41253
I20260928 16:40:17.094965 25125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:41253 every 8 connection(s)
I20260928 16:40:17.096863 25058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:40:17.101709 25126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:17.106434 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 25058
I20260928 16:40:17.106684 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:40:17.107681 11231 external_mini_cluster.cc:1428] Setting key 04b768e93e594e074b76a37ba47dde54
I20260928 16:40:17.117038 25126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a: Bootstrap starting.
I20260928 16:40:17.121704 25126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:17.123260 25126 log.cc:824] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:17.127214 25126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a: No bootstrap required, opened a new log
Sep 28 16:40:17 dist-test-slave-9fn6 krb5kdc[25045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:17.138765 25126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf3512ffad94903b83ccea9581f581a" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41253 } }
I20260928 16:40:17.139344 25126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:17.139467 25126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf3512ffad94903b83ccea9581f581a, State: Initialized, Role: FOLLOWER
I20260928 16:40:17.139685 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:17.110234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49996 (local address 127.10.247.254:41253)
0928 16:40:17.111216 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:17.111236 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:17.111302 (+ 66us) server_negotiation.cc:407] Connection header received
0928 16:40:17.114705 (+ 3403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:17.114750 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:17.115468 (+ 718us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:17.116065 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:17.118143 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:17.120362 (+ 2219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:17.122283 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:17.123482 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:17.123703 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:17.129160 (+ 5457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:17.129177 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:17.129205 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:17.129273 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:17.133806 (+ 4533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:17.134784 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:17.134797 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:17.134808 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:17.135069 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:17.135476 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:17.135481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:17.135485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:17.136462 (+ 977us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:17.136893 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:17.137392 (+ 499us) server_negotiation.cc:299] Negotiation successful
0928 16:40:17.138005 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":417,"threads_started":1}
I20260928 16:40:17.140280 25126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf3512ffad94903b83ccea9581f581a" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41253 } }
I20260928 16:40:17.140571 25126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:40:17.140790 25126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:40:17.140977 25126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:17.143819 25126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf3512ffad94903b83ccea9581f581a" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41253 } }
I20260928 16:40:17.144491 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:40:17.144443 25126 leader_election.cc:302] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cf3512ffad94903b83ccea9581f581a; no voters:
I20260928 16:40:17.145171 25126 leader_election.cc:288] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:40:17.145366 25131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:40:17.147554 25126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:40:17.148386 25131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [term 1 LEADER]: Becoming Leader. State: Replica: 3cf3512ffad94903b83ccea9581f581a, State: Running, Role: LEADER
I20260928 16:40:17.149264 25131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf3512ffad94903b83ccea9581f581a" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41253 } }
I20260928 16:40:17.152966 25132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cf3512ffad94903b83ccea9581f581a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf3512ffad94903b83ccea9581f581a" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41253 } } }
I20260928 16:40:17.153486 25132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:17.157159 25133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cf3512ffad94903b83ccea9581f581a. Latest consensus state: current_term: 1 leader_uuid: "3cf3512ffad94903b83ccea9581f581a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf3512ffad94903b83ccea9581f581a" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41253 } } }
I20260928 16:40:17.157703 25133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:17.158295 25140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:40:17.163607 25140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:40:17.173553 25140 catalog_manager.cc:1347] Generated new cluster ID: 2d502d6b50d941e6b119451a13933f2c
I20260928 16:40:17.173709 25140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:40:17.195225 25140 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:40:17.197119 25140 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:40:17.204762 25140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cf3512ffad94903b83ccea9581f581a: Generated new TSK 0
I20260928 16:40:17.205926 25140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:40:17.260200 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 25058
I20260928 16:40:17.261610 25121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:17.369551 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 25058
2026-09-28T16:40:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2159 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:40:17 dist-test-slave-9fn6 krb5kdc[25154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:40:17 dist-test-slave-9fn6 krb5kdc[25154](info): set up 2 sockets
Sep 28 16:40:17 dist-test-slave-9fn6 krb5kdc[25154](info): commencing operation
krb5kdc: starting...
W20260928 16:40:18.429272 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.009s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:40:18 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:40:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:40:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:40:18.730118 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45029
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:45029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:19.126209 25167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:19.126612 25167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:19.126744 25167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:19.134907 25167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:19.135128 25167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:19.135195 25167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:19.135279 25167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:19.135347 25167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:19.146562 25167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:45029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:19.148387 25167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:19.150120 25167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:19.163769 25172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:19.163736 25175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:19.163729 25173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:19.164649 25167 server_base.cc:1034] running on GCE node
I20260928 16:40:19.165655 25167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:19.167531 25167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:19.168728 25167 hybrid_clock.cc:648] HybridClock initialized: now 1790613619168678 us; error 37 us; skew 500 ppm
I20260928 16:40:19.169185 25167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:19 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613619, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:19.172906 25167 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:40:19.174902 25167 webserver.cc:460] Webserver started at http://127.10.247.254:40239/ using document root <none> and password file <none>
I20260928 16:40:19.175817 25167 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:19.175961 25167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:19.176337 25167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:19.180338 25167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "416a40b028794fcdabeff6550154ebed"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "f5b761d47e1606a8e9e9da76c0bd4652"
server_key_iv: "50fcaa86c5e860cebc4680facab09d38"
server_key_version: "encryptionkey@0"
I20260928 16:40:19.181326 25167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "416a40b028794fcdabeff6550154ebed"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "f5b761d47e1606a8e9e9da76c0bd4652"
server_key_iv: "50fcaa86c5e860cebc4680facab09d38"
server_key_version: "encryptionkey@0"
I20260928 16:40:19.187901 25167 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:40:19.192468 25182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:19.194226 25167 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:40:19.194423 25167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "416a40b028794fcdabeff6550154ebed"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "f5b761d47e1606a8e9e9da76c0bd4652"
server_key_iv: "50fcaa86c5e860cebc4680facab09d38"
server_key_version: "encryptionkey@0"
I20260928 16:40:19.194648 25167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:19.230154 25167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:19.235306 25167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:19.235623 25167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:19.256294 25167 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:45029
I20260928 16:40:19.256318 25235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:45029 every 8 connection(s)
I20260928 16:40:19.258013 25167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:40:19.262676 25236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:19.263105 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 25167
I20260928 16:40:19.263393 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:40:19.264416 11231 external_mini_cluster.cc:1428] Setting key df9d4bfe543c2c82c3c3f05cea976c78
I20260928 16:40:19.280407 25236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Bootstrap starting.
Sep 28 16:40:19 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:19.285355 25236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:19.286893 25236 log.cc:824] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:19.291208 25236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: No bootstrap required, opened a new log
I20260928 16:40:19.292860 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:19.267085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.10.247.254:45029)
0928 16:40:19.268055 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:19.268070 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:19.268126 (+ 56us) server_negotiation.cc:407] Connection header received
0928 16:40:19.270360 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:19.270392 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:19.270981 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:19.271446 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:19.273826 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:19.275389 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:19.277499 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:19.278349 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:19.278502 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:19.284307 (+ 5805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:19.284317 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:19.284335 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:19.284387 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:19.287956 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:19.288901 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:19.288909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:19.288918 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:19.289112 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:19.289583 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:19.289586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:19.289588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:19.290206 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:19.290476 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:19.291034 (+ 558us) server_negotiation.cc:299] Negotiation successful
0928 16:40:19.291523 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":409,"threads_started":1}
I20260928 16:40:19.300921 25236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:19.301506 25236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:19.301631 25236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 416a40b028794fcdabeff6550154ebed, State: Initialized, Role: FOLLOWER
I20260928 16:40:19.302315 25236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:19.302582 25236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:40:19.302731 25236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:40:19.302893 25236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:19.305786 25236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:19.306432 25236 leader_election.cc:302] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 416a40b028794fcdabeff6550154ebed; no voters:
I20260928 16:40:19.307145 25236 leader_election.cc:288] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:40:19.307302 25241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:40:19.308375 25241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 1 LEADER]: Becoming Leader. State: Replica: 416a40b028794fcdabeff6550154ebed, State: Running, Role: LEADER
I20260928 16:40:19.309124 25241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:19.309798 25236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:40:19.312346 25243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 416a40b028794fcdabeff6550154ebed. Latest consensus state: current_term: 1 leader_uuid: "416a40b028794fcdabeff6550154ebed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } } }
I20260928 16:40:19.313084 25243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:19.312899 25242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "416a40b028794fcdabeff6550154ebed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } } }
I20260928 16:40:19.313235 25242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:19.315208 25250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:40:19.321657 25250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:40:19.333086 25250 catalog_manager.cc:1347] Generated new cluster ID: 73715f95e3fc44b28e4302e79ea8c648
I20260928 16:40:19.333314 25250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:40:19.351717 25250 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:40:19.354071 25250 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:40:19.378553 25250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Generated new TSK 0
I20260928 16:40:19.380019 25250 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:40:19.450687 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45029
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:19.839069 25262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:19.839460 25262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:19.839610 25262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:19.847620 25262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:19.847785 25262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:19.847965 25262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:40:19.859719 25262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:19.862007 25262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:19.864328 25262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:19.882238 25268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:19.882596 25267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:19.883174 25262 server_base.cc:1034] running on GCE node
W20260928 16:40:19.883665 25270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:19.884676 25262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:19.886123 25262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:19.887434 25262 hybrid_clock.cc:648] HybridClock initialized: now 1790613619887349 us; error 69 us; skew 500 ppm
I20260928 16:40:19.887861 25262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:19 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613619, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:19.892781 25262 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:40:19.898329 25262 webserver.cc:460] Webserver started at http://127.10.247.193:36287/ using document root <none> and password file <none>
I20260928 16:40:19.899389 25262 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:19.899580 25262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:19.899960 25262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:19.903882 25262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "1659addd8a214901b259e07421fc8581"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "4374d8add0dc0b7ca3a7376c1ea5c834"
server_key_iv: "ad32b90253cd7df16c8c1dd02168936b"
server_key_version: "encryptionkey@0"
I20260928 16:40:19.904971 25262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "1659addd8a214901b259e07421fc8581"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "4374d8add0dc0b7ca3a7376c1ea5c834"
server_key_iv: "ad32b90253cd7df16c8c1dd02168936b"
server_key_version: "encryptionkey@0"
I20260928 16:40:19.912030 25262 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260928 16:40:19.916384 25277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:19.918321 25262 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260928 16:40:19.918548 25262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "1659addd8a214901b259e07421fc8581"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "4374d8add0dc0b7ca3a7376c1ea5c834"
server_key_iv: "ad32b90253cd7df16c8c1dd02168936b"
server_key_version: "encryptionkey@0"
I20260928 16:40:19.918792 25262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:19.951156 25262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:19.956748 25262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:19.957095 25262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:19.958317 25262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:19.960058 25262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:19.960145 25262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:19.960280 25262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:19.960336 25262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:19.994899 25262 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:37211
I20260928 16:40:19.994932 25389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:37211 every 8 connection(s)
I20260928 16:40:19.996461 25262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:40:20.006942 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 25262
I20260928 16:40:20.007182 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:40:20.007879 11231 external_mini_cluster.cc:1428] Setting key 695ef287faf62156898d1d46348fe21e
Sep 28 16:40:20 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613619, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:20.020674 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:19.999710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:55497 (local address 127.10.247.254:45029)
0928 16:40:20.000245 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:20.000251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:20.001491 (+ 1240us) server_negotiation.cc:407] Connection header received
0928 16:40:20.003559 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:20.003566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:20.003677 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:20.003787 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:20.005675 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:20.006577 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:20.008523 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:20.009196 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:20.009308 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:20.013960 (+ 4652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:20.013966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:20.013970 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:20.014008 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:20.016897 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:20.017881 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:20.017887 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:20.017889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:20.018074 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:20.018563 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:20.018568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:20.018570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:20.018910 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:20.019076 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:20.020071 (+ 995us) server_negotiation.cc:299] Negotiation successful
0928 16:40:20.020324 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":263,"threads_started":1}
I20260928 16:40:20.022162 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:20.000398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:55497)
0928 16:40:20.001299 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:20.001344 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:20.003247 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:20.004070 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:20.004080 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:20.004767 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:20.005477 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:20.005497 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:20.006795 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:20.006799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:20.008352 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:20.008367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:20.009463 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:20.009471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:20.009833 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:20.010739 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:20.010765 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:20.013698 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:20.017064 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:20.017076 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:20.017100 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:20.017744 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:20.018175 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:20.018180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:20.018183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:20.018450 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:20.019066 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:20.019076 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:20.019597 (+ 521us) client_negotiation.cc:769] Sending connection context
0928 16:40:20.019919 (+ 322us) client_negotiation.cc:240] Negotiation successful
0928 16:40:20.020562 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":364,"threads_started":1}
I20260928 16:40:20.024394 25390 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45029
I20260928 16:40:20.024819 25390 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:20.025775 25390 heartbeater.cc:511] Master 127.10.247.254:45029 requested a full tablet report, sending...
I20260928 16:40:20.028681 25199 ts_manager.cc:194] Registered new tserver with Master: 1659addd8a214901b259e07421fc8581 (127.10.247.193:37211)
I20260928 16:40:20.030925 25199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:55497
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:40:20.076589 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45029
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:20.465179 25396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:20.465533 25396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:20.465669 25396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:20.475695 25396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:20.475872 25396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:20.476085 25396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:40:20.487731 25396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:20.489527 25396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:20.491170 25396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:20.506305 25401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:20.506347 25402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:20.506362 25404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:20.507290 25396 server_base.cc:1034] running on GCE node
I20260928 16:40:20.508092 25396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:20.509291 25396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:20.510492 25396 hybrid_clock.cc:648] HybridClock initialized: now 1790613620510420 us; error 62 us; skew 500 ppm
I20260928 16:40:20.510859 25396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:20 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613620, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:20.517664 25396 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:40:20.519416 25396 webserver.cc:460] Webserver started at http://127.10.247.194:37731/ using document root <none> and password file <none>
I20260928 16:40:20.520273 25396 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:20.520402 25396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:20.520783 25396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:20.524456 25396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "71fbefddafda49f18658dd24387d4987"
format_stamp: "Formatted at 2026-09-28 16:40:20 on dist-test-slave-9fn6"
server_key: "d5c7bd288213c4123f4cdd37bfe83623"
server_key_iv: "236e8e6d47ebebfe8a89e9004c8507e8"
server_key_version: "encryptionkey@0"
I20260928 16:40:20.525396 25396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "71fbefddafda49f18658dd24387d4987"
format_stamp: "Formatted at 2026-09-28 16:40:20 on dist-test-slave-9fn6"
server_key: "d5c7bd288213c4123f4cdd37bfe83623"
server_key_iv: "236e8e6d47ebebfe8a89e9004c8507e8"
server_key_version: "encryptionkey@0"
I20260928 16:40:20.531836 25396 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260928 16:40:20.536075 25411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:20.537850 25396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:40:20.538041 25396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "71fbefddafda49f18658dd24387d4987"
format_stamp: "Formatted at 2026-09-28 16:40:20 on dist-test-slave-9fn6"
server_key: "d5c7bd288213c4123f4cdd37bfe83623"
server_key_iv: "236e8e6d47ebebfe8a89e9004c8507e8"
server_key_version: "encryptionkey@0"
I20260928 16:40:20.538264 25396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:20.563746 25396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:20.568966 25396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:20.569281 25396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:20.570484 25396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:20.572180 25396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:20.572273 25396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:20.572408 25396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:20.572466 25396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:20.607993 25396 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:46121
I20260928 16:40:20.608031 25523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:46121 every 8 connection(s)
I20260928 16:40:20.609656 25396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:40:20.618870 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 25396
I20260928 16:40:20.619073 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:40:20.619599 11231 external_mini_cluster.cc:1428] Setting key ffed9702a839ee381566f71d95c21c09
Sep 28 16:40:20 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613620, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:20.634470 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:20.612756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:49179 (local address 127.10.247.254:45029)
0928 16:40:20.613277 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:20.613282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:20.614415 (+ 1133us) server_negotiation.cc:407] Connection header received
0928 16:40:20.616650 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:20.616657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:20.616785 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:20.616898 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:20.618833 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:20.619729 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:20.621984 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:20.622571 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:20.622739 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:20.627363 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:20.627370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:20.627374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:20.627421 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:20.630373 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:20.631522 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:20.631527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:20.631530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:20.631669 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:20.632156 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:20.632159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:20.632161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:20.632495 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:20.632665 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:20.633736 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0928 16:40:20.634031 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":240,"threads_started":1}
I20260928 16:40:20.635576 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:20.613376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:49179)
0928 16:40:20.614204 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:20.614257 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:20.616270 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:20.617204 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:20.617214 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:20.617856 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:20.618612 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:20.618640 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:20.619897 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:20.619901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:20.621807 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:20.621826 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:20.622859 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:20.622868 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:20.623142 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:20.623982 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:20.624024 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:20.626996 (+ 2972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:20.630595 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:20.630608 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:20.630636 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:20.631395 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:20.631763 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:20.631768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:20.631771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:20.632060 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:20.632644 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:20.632654 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:20.633232 (+ 578us) client_negotiation.cc:769] Sending connection context
0928 16:40:20.633568 (+ 336us) client_negotiation.cc:240] Negotiation successful
0928 16:40:20.634117 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":301,"threads_started":1}
I20260928 16:40:20.637400 25524 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45029
I20260928 16:40:20.637797 25524 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:20.638854 25524 heartbeater.cc:511] Master 127.10.247.254:45029 requested a full tablet report, sending...
I20260928 16:40:20.640938 25199 ts_manager.cc:194] Registered new tserver with Master: 71fbefddafda49f18658dd24387d4987 (127.10.247.194:46121)
I20260928 16:40:20.641933 25199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:49179
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:40:20.688330 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45029
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:40:21.034845 25390 heartbeater.cc:503] Master 127.10.247.254:45029 was elected leader, sending a full tablet report...
W20260928 16:40:21.079483 25530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:21.079844 25530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:21.080005 25530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:21.087929 25530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:21.088069 25530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:21.088233 25530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:40:21.100144 25530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:21.101961 25530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:21.103574 25530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:21.118803 25536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:21.118891 25538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:21.118822 25535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:21.120112 25530 server_base.cc:1034] running on GCE node
I20260928 16:40:21.120919 25530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:21.122118 25530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:21.123358 25530 hybrid_clock.cc:648] HybridClock initialized: now 1790613621123292 us; error 45 us; skew 500 ppm
I20260928 16:40:21.123745 25530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:21 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613621, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:21.127478 25530 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:40:21.129381 25530 webserver.cc:460] Webserver started at http://127.10.247.195:36707/ using document root <none> and password file <none>
I20260928 16:40:21.130259 25530 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:21.130399 25530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:21.130749 25530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:21.134527 25530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "45b25079458041dfb0590947818682a1"
format_stamp: "Formatted at 2026-09-28 16:40:21 on dist-test-slave-9fn6"
server_key: "eccd213a67c76a8767fcdf0ec20a2bc5"
server_key_iv: "d655050514553e54f546b5cb23e81244"
server_key_version: "encryptionkey@0"
I20260928 16:40:21.135476 25530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "45b25079458041dfb0590947818682a1"
format_stamp: "Formatted at 2026-09-28 16:40:21 on dist-test-slave-9fn6"
server_key: "eccd213a67c76a8767fcdf0ec20a2bc5"
server_key_iv: "d655050514553e54f546b5cb23e81244"
server_key_version: "encryptionkey@0"
I20260928 16:40:21.141937 25530 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260928 16:40:21.146262 25545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:21.147984 25530 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:40:21.148175 25530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "45b25079458041dfb0590947818682a1"
format_stamp: "Formatted at 2026-09-28 16:40:21 on dist-test-slave-9fn6"
server_key: "eccd213a67c76a8767fcdf0ec20a2bc5"
server_key_iv: "d655050514553e54f546b5cb23e81244"
server_key_version: "encryptionkey@0"
I20260928 16:40:21.148417 25530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:21.166740 25530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:21.171871 25530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:21.172215 25530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:21.173471 25530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:21.175156 25530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:21.175244 25530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:21.175370 25530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:21.175424 25530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:21.208287 25530 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:43495
I20260928 16:40:21.208333 25657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:43495 every 8 connection(s)
I20260928 16:40:21.209868 25530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:40:21.210309 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 25530
I20260928 16:40:21.210552 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:40:21.211236 11231 external_mini_cluster.cc:1428] Setting key c6e70b104ded40ad4dd6f524e82001ef
Sep 28 16:40:21 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613621, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:21.233124 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.213124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:60519 (local address 127.10.247.254:45029)
0928 16:40:21.213709 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.213715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.214996 (+ 1281us) server_negotiation.cc:407] Connection header received
0928 16:40:21.217247 (+ 2251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.217252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.217360 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.217463 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:21.219253 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.220122 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.221824 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.222294 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.222417 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.226379 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:21.226385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:21.226388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:21.226431 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:21.229807 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.230624 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.230629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.230631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.230786 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.231222 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.231226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.231228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.231573 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:21.231714 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.232498 (+ 784us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.232751 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":307,"threads_started":1}
I20260928 16:40:21.234282 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.213864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:60519)
0928 16:40:21.214818 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:21.214873 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:21.216946 (+ 2073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:21.217761 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:21.217771 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:21.218372 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:21.219070 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.219089 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.220291 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.220295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.221680 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.221695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.222541 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.222551 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.222828 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.223525 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:21.223556 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:21.226109 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:21.229958 (+ 3849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:21.229967 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:21.229985 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:21.230488 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:21.230869 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:21.230872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:21.230875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:21.231086 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:21.231696 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:21.231703 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:21.232121 (+ 418us) client_negotiation.cc:769] Sending connection context
0928 16:40:21.232386 (+ 265us) client_negotiation.cc:240] Negotiation successful
0928 16:40:21.232942 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":310,"threads_started":1}
I20260928 16:40:21.236052 25658 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45029
I20260928 16:40:21.236428 25658 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:21.237381 25658 heartbeater.cc:511] Master 127.10.247.254:45029 requested a full tablet report, sending...
I20260928 16:40:21.239118 25199 ts_manager.cc:194] Registered new tserver with Master: 45b25079458041dfb0590947818682a1 (127.10.247.195:43495)
I20260928 16:40:21.240317 25199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:60519
I20260928 16:40:21.242026 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:40:21.262007 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.247643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.10.247.254:45029)
0928 16:40:21.247852 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.247859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.248347 (+ 488us) server_negotiation.cc:407] Connection header received
0928 16:40:21.248550 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.248557 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.248698 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.248794 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:21.250367 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.251208 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.252649 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.253107 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.253226 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.255915 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:21.255923 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:21.255929 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:21.255978 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:21.258930 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.259717 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.259724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.259727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.259876 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.260276 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.260282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.260284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.260757 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:21.261015 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.261384 (+ 369us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.261658 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 16:40:21.266995 25199 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:40:21.274791 25199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:40:21.302237 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.289866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.10.247.194:46121)
0928 16:40:21.291620 (+ 1754us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.291630 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.291661 (+ 31us) server_negotiation.cc:407] Connection header received
0928 16:40:21.291792 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.291801 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.292127 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.294036 (+ 1909us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:21.294389 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.295792 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.299140 (+ 3348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.300739 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.301362 (+ 623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.301510 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.301595 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.301850 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1587,"thread_start_us":1428,"threads_started":1}
I20260928 16:40:21.302271 25671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.289628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:46121 (local address 127.0.0.1:60582)
0928 16:40:21.290674 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:21.290738 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:21.290966 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:21.292397 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:21.292406 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:21.292468 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:21.293434 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.293450 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.295921 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.295926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.298921 (+ 2995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.298936 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.301013 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.301034 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.301243 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.301337 (+ 94us) client_negotiation.cc:769] Sending connection context
0928 16:40:21.301466 (+ 129us) client_negotiation.cc:240] Negotiation successful
0928 16:40:21.301729 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":73,"spinlock_wait_cycles":21888,"thread_start_us":752,"threads_started":1}
I20260928 16:40:21.302956 25673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.290177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:39540)
0928 16:40:21.292343 (+ 2166us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:21.292360 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:21.292466 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:21.293776 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:21.293781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:21.293809 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:21.294233 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.294245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.296138 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.296144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.298750 (+ 2606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.298763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.301052 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.301060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.301849 (+ 789us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.301888 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 16:40:21.301977 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 16:40:21.302122 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2025,"mutex_wait_us":132,"thread_start_us":1836,"threads_started":1}
I20260928 16:40:21.303138 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.292993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.10.247.195:43495)
0928 16:40:21.294495 (+ 1502us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.294503 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.294541 (+ 38us) server_negotiation.cc:407] Connection header received
0928 16:40:21.294651 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.294658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.294881 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.295090 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:21.295680 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.297321 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.299911 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.301363 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.301561 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.301750 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.302616 (+ 866us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.302788 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1319,"thread_start_us":1121,"threads_started":1}
I20260928 16:40:21.303272 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.290561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39540 (local address 127.10.247.193:37211)
0928 16:40:21.293083 (+ 2522us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.293092 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.293121 (+ 29us) server_negotiation.cc:407] Connection header received
0928 16:40:21.293242 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.293250 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.293523 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.293720 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:21.294451 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.295943 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.298923 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.300345 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.300568 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.300825 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.302397 (+ 1572us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.302578 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2328,"thread_start_us":2153,"threads_started":1}
I20260928 16:40:21.303534 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.292339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:43495 (local address 127.0.0.1:41892)
0928 16:40:21.293388 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:21.293409 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:21.293556 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:21.295103 (+ 1547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:21.295107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:21.295132 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:21.295500 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.295511 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.297483 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.297492 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.299728 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.299742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.301602 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.301608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.301788 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.302278 (+ 490us) client_negotiation.cc:769] Sending connection context
0928 16:40:21.302369 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 16:40:21.302506 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":428,"thread_start_us":1026,"threads_started":1}
I20260928 16:40:21.307581 25593 tablet_service.cc:1467] Processing CreateTablet for tablet 81d5a05f075f4e02879de5d79930989c (DEFAULT_TABLE table=test-table [id=4d9f53cdce5a48d8b35fd24c87a86d04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:21.307574 25459 tablet_service.cc:1467] Processing CreateTablet for tablet 81d5a05f075f4e02879de5d79930989c (DEFAULT_TABLE table=test-table [id=4d9f53cdce5a48d8b35fd24c87a86d04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:21.307585 25325 tablet_service.cc:1467] Processing CreateTablet for tablet 81d5a05f075f4e02879de5d79930989c (DEFAULT_TABLE table=test-table [id=4d9f53cdce5a48d8b35fd24c87a86d04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:21.309158 25459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d5a05f075f4e02879de5d79930989c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:21.309175 25593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d5a05f075f4e02879de5d79930989c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:21.309192 25325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d5a05f075f4e02879de5d79930989c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:21.322834 25677 tablet_bootstrap.cc:492] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: Bootstrap starting.
I20260928 16:40:21.327905 25677 tablet_bootstrap.cc:654] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:21.328999 25679 tablet_bootstrap.cc:492] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: Bootstrap starting.
I20260928 16:40:21.329561 25677 log.cc:824] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:21.330056 25678 tablet_bootstrap.cc:492] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: Bootstrap starting.
I20260928 16:40:21.334015 25679 tablet_bootstrap.cc:654] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:21.334007 25678 tablet_bootstrap.cc:654] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:21.334563 25677 tablet_bootstrap.cc:492] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: No bootstrap required, opened a new log
I20260928 16:40:21.334899 25677 ts_tablet_manager.cc:1397] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260928 16:40:21.335353 25679 log.cc:824] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:21.335434 25678 log.cc:824] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:21.338810 25678 tablet_bootstrap.cc:492] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: No bootstrap required, opened a new log
I20260928 16:40:21.338892 25679 tablet_bootstrap.cc:492] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: No bootstrap required, opened a new log
I20260928 16:40:21.339119 25678 ts_tablet_manager.cc:1397] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260928 16:40:21.339197 25679 ts_tablet_manager.cc:1397] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260928 16:40:21.346493 25677 raft_consensus.cc:348] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:21.347112 25677 raft_consensus.cc:374] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:21.347229 25677 raft_consensus.cc:729] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71fbefddafda49f18658dd24387d4987, State: Initialized, Role: FOLLOWER
I20260928 16:40:21.347780 25678 raft_consensus.cc:348] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:21.348181 25678 raft_consensus.cc:374] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:21.348034 25677 consensus_queue.cc:260] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:21.348286 25678 raft_consensus.cc:729] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1659addd8a214901b259e07421fc8581, State: Initialized, Role: FOLLOWER
I20260928 16:40:21.348935 25678 consensus_queue.cc:260] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:21.351085 25678 ts_tablet_manager.cc:1428] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260928 16:40:21.350752 25679 raft_consensus.cc:348] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:21.351305 25679 raft_consensus.cc:374] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:21.351397 25679 raft_consensus.cc:729] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b25079458041dfb0590947818682a1, State: Initialized, Role: FOLLOWER
I20260928 16:40:21.352216 25679 consensus_queue.cc:260] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:21.355691 25524 heartbeater.cc:503] Master 127.10.247.254:45029 was elected leader, sending a full tablet report...
I20260928 16:40:21.357169 25677 ts_tablet_manager.cc:1428] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: Time spent starting tablet: real 0.022s user 0.010s sys 0.013s
I20260928 16:40:21.358239 25658 heartbeater.cc:503] Master 127.10.247.254:45029 was elected leader, sending a full tablet report...
I20260928 16:40:21.358964 25679 ts_tablet_manager.cc:1428] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: Time spent starting tablet: real 0.020s user 0.013s sys 0.008s
W20260928 16:40:21.361860 25525 tablet.cc:2369] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:40:21.367439 25684 raft_consensus.cc:482] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:40:21.367769 25684 raft_consensus.cc:504] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:21.369999 25684 leader_election.cc:288] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71fbefddafda49f18658dd24387d4987 (127.10.247.194:46121), 45b25079458041dfb0590947818682a1 (127.10.247.195:43495)
I20260928 16:40:21.378286 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.370438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:39739 (local address 127.10.247.194:46121)
0928 16:40:21.370713 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.370720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.371415 (+ 695us) server_negotiation.cc:407] Connection header received
0928 16:40:21.371640 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.371645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.371719 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.371800 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:21.372437 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.373253 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.375785 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.377115 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.377283 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.377369 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.377811 (+ 442us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.377966 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 16:40:21.378286 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.370706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:34973 (local address 127.10.247.195:43495)
0928 16:40:21.370978 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.370985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.371646 (+ 661us) server_negotiation.cc:407] Connection header received
0928 16:40:21.371766 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.371770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.371835 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.371912 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:21.372563 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.373429 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.375743 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.377120 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.377286 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.377369 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.377806 (+ 437us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.377966 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260928 16:40:21.378486 25686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.370292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:46121 (local address 127.10.247.193:39739)
0928 16:40:21.371282 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:21.371303 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:21.371488 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:21.371831 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:21.371836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:21.371861 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:21.372275 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.372290 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.373387 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.373392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.375489 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.375506 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.377370 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.377378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.377634 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.377660 (+ 26us) client_negotiation.cc:769] Sending connection context
0928 16:40:21.377758 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 16:40:21.377970 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":163,"thread_start_us":630,"threads_started":1}
I20260928 16:40:21.378509 25687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.370540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:43495 (local address 127.10.247.193:34973)
0928 16:40:21.371484 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:21.371499 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:21.371643 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:21.371944 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:21.371948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:21.371993 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:21.372406 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.372416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.373575 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.373579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.375519 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:21.375531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.377390 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.377396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:21.377618 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.377658 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 16:40:21.377760 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 16:40:21.377970 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":88,"thread_start_us":627,"threads_started":1}
I20260928 16:40:21.379773 25479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d5a05f075f4e02879de5d79930989c" candidate_uuid: "1659addd8a214901b259e07421fc8581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71fbefddafda49f18658dd24387d4987" is_pre_election: true
I20260928 16:40:21.379982 25613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d5a05f075f4e02879de5d79930989c" candidate_uuid: "1659addd8a214901b259e07421fc8581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b25079458041dfb0590947818682a1" is_pre_election: true
I20260928 16:40:21.380487 25479 raft_consensus.cc:2454] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1659addd8a214901b259e07421fc8581 in term 0.
I20260928 16:40:21.380513 25613 raft_consensus.cc:2454] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1659addd8a214901b259e07421fc8581 in term 0.
I20260928 16:40:21.381242 25281 leader_election.cc:302] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1659addd8a214901b259e07421fc8581, 45b25079458041dfb0590947818682a1; no voters:
I20260928 16:40:21.381695 25684 raft_consensus.cc:2781] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:40:21.381846 25684 raft_consensus.cc:482] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:40:21.381948 25684 raft_consensus.cc:3037] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:21.384466 25684 raft_consensus.cc:504] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:21.385255 25684 leader_election.cc:288] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [CANDIDATE]: Term 1 election: Requested vote from peers 71fbefddafda49f18658dd24387d4987 (127.10.247.194:46121), 45b25079458041dfb0590947818682a1 (127.10.247.195:43495)
I20260928 16:40:21.385838 25613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d5a05f075f4e02879de5d79930989c" candidate_uuid: "1659addd8a214901b259e07421fc8581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b25079458041dfb0590947818682a1"
I20260928 16:40:21.385864 25479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d5a05f075f4e02879de5d79930989c" candidate_uuid: "1659addd8a214901b259e07421fc8581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71fbefddafda49f18658dd24387d4987"
I20260928 16:40:21.386072 25613 raft_consensus.cc:3037] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:21.386102 25479 raft_consensus.cc:3037] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:21.388661 25613 raft_consensus.cc:2454] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1659addd8a214901b259e07421fc8581 in term 1.
I20260928 16:40:21.388731 25479 raft_consensus.cc:2454] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1659addd8a214901b259e07421fc8581 in term 1.
I20260928 16:40:21.389161 25281 leader_election.cc:302] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1659addd8a214901b259e07421fc8581, 45b25079458041dfb0590947818682a1; no voters:
I20260928 16:40:21.389498 25684 raft_consensus.cc:2781] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:40:21.389984 25684 raft_consensus.cc:686] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 1 LEADER]: Becoming Leader. State: Replica: 1659addd8a214901b259e07421fc8581, State: Running, Role: LEADER
I20260928 16:40:21.390610 25684 consensus_queue.cc:237] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:21.395943 25197 catalog_manager.cc:5495] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1659addd8a214901b259e07421fc8581 (127.10.247.193). New cstate: current_term: 1 leader_uuid: "1659addd8a214901b259e07421fc8581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260928 16:40:21.434543 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.428920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39552 (local address 127.10.247.193:37211)
0928 16:40:21.429133 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.429139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.429230 (+ 91us) server_negotiation.cc:407] Connection header received
0928 16:40:21.429449 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.429453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.429550 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.429687 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:40:21.430042 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.430817 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.432280 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.432766 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.432862 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.433364 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:40:21.434023 (+ 659us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:40:21.434153 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.434202 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.434285 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 16:40:21.444780 25479 raft_consensus.cc:1261] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [term 1 FOLLOWER]: Refusing update from remote peer 1659addd8a214901b259e07421fc8581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:21.444797 25613 raft_consensus.cc:1261] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [term 1 FOLLOWER]: Refusing update from remote peer 1659addd8a214901b259e07421fc8581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:21.445937 25684 consensus_queue.cc:1035] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:40:21.446275 25688 consensus_queue.cc:1035] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:40:21.475595 25197 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:40:21.475950 25197 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:40:21.480392 25197 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Sending DeleteTablet for 3 replicas of tablet 81d5a05f075f4e02879de5d79930989c
I20260928 16:40:21.481984 25459 tablet_service.cc:1514] Processing DeleteTablet for tablet 81d5a05f075f4e02879de5d79930989c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:21 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:60582
I20260928 16:40:21.482282 25325 tablet_service.cc:1514] Processing DeleteTablet for tablet 81d5a05f075f4e02879de5d79930989c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:21 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:39540
I20260928 16:40:21.482337 25593 tablet_service.cc:1514] Processing DeleteTablet for tablet 81d5a05f075f4e02879de5d79930989c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:21 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:41892
I20260928 16:40:21.483381 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 25262
I20260928 16:40:21.495179 25702 tablet_replica.cc:330] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: stopping tablet replica
I20260928 16:40:21.495405 25701 tablet_replica.cc:330] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: stopping tablet replica
I20260928 16:40:21.496156 25703 tablet_replica.cc:330] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: stopping tablet replica
I20260928 16:40:21.496155 25701 raft_consensus.cc:2229] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:40:21.496851 25703 raft_consensus.cc:2229] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:40:21.496938 25701 raft_consensus.cc:2258] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:21.497545 25702 raft_consensus.cc:2229] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:40:21.499071 25702 raft_consensus.cc:2258] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:21.499835 25701 ts_tablet_manager.cc:1905] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:21.499922 25703 raft_consensus.cc:2258] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:21.502876 25702 ts_tablet_manager.cc:1905] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:21.502882 25703 ts_tablet_manager.cc:1905] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 16:40:21 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:40:21.511101 25703 ts_tablet_manager.cc:1918] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:21.511545 25703 log.cc:1196] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/81d5a05f075f4e02879de5d79930989c
I20260928 16:40:21.512310 25703 ts_tablet_manager.cc:1939] T 81d5a05f075f4e02879de5d79930989c P 1659addd8a214901b259e07421fc8581: Deleting consensus metadata
I20260928 16:40:21.514969 25186 catalog_manager.cc:4841] TS 1659addd8a214901b259e07421fc8581 (127.10.247.193:37211): tablet 81d5a05f075f4e02879de5d79930989c (table test-table [id=4d9f53cdce5a48d8b35fd24c87a86d04]) successfully deleted
I20260928 16:40:21.515350 25701 ts_tablet_manager.cc:1918] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:21.515587 25701 log.cc:1196] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/81d5a05f075f4e02879de5d79930989c
I20260928 16:40:21.516249 25701 ts_tablet_manager.cc:1939] T 81d5a05f075f4e02879de5d79930989c P 71fbefddafda49f18658dd24387d4987: Deleting consensus metadata
I20260928 16:40:21.518610 25183 catalog_manager.cc:4841] TS 71fbefddafda49f18658dd24387d4987 (127.10.247.194:46121): tablet 81d5a05f075f4e02879de5d79930989c (table test-table [id=4d9f53cdce5a48d8b35fd24c87a86d04]) successfully deleted
I20260928 16:40:21.519002 25702 ts_tablet_manager.cc:1918] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:21.519269 25702 log.cc:1196] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/81d5a05f075f4e02879de5d79930989c
I20260928 16:40:21.519896 25702 ts_tablet_manager.cc:1939] T 81d5a05f075f4e02879de5d79930989c P 45b25079458041dfb0590947818682a1: Deleting consensus metadata
I20260928 16:40:21.520017 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.484497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39568 (local address 127.10.247.193:37211)
0928 16:40:21.484739 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.484747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.494100 (+ 9353us) server_negotiation.cc:407] Connection header received
0928 16:40:21.494516 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.494524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.494726 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.494837 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:21.497556 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.498889 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.501399 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.502153 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.502278 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.509480 (+ 7202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:21.509501 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:21.509528 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:21.509606 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:21.514748 (+ 5142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.516147 (+ 1399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.516160 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.516164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.516390 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.517069 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.517074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.517079 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.517652 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:21.518052 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.519317 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.519610 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:40:21.520893 25385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:21.522003 25186 catalog_manager.cc:4841] TS 45b25079458041dfb0590947818682a1 (127.10.247.195:43495): tablet 81d5a05f075f4e02879de5d79930989c (table test-table [id=4d9f53cdce5a48d8b35fd24c87a86d04]) successfully deleted
I20260928 16:40:21.642416 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 25262
I20260928 16:40:21.662132 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 25396
Sep 28 16:40:21 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:40:21.678769 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.663130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37790 (local address 127.10.247.194:46121)
0928 16:40:21.663343 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.663349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.663372 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:40:21.663569 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.663575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.663663 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.663774 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:21.665283 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.666096 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.667437 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.667896 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.668012 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.672538 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:21.672557 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:21.672578 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:21.672643 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:21.675512 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.676266 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.676276 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.676279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.676422 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.676962 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.676967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.676971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.677457 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:21.677784 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.678127 (+ 343us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.678389 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 16:40:21.679607 25519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:21.775266 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 25396
I20260928 16:40:21.791971 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 25530
Sep 28 16:40:21 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:40:21.809288 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:21.792860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52708 (local address 127.10.247.195:43495)
0928 16:40:21.793102 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:21.793107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:21.793129 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:40:21.793329 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:21.793334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:21.793414 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:21.793502 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:21.795186 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.796064 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.797532 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:21.798010 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:21.798117 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:21.803090 (+ 4973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:21.803111 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:21.803133 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:21.803186 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:21.806291 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.807022 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.807033 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.807035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.807211 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:21.807601 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:21.807606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:21.807608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:21.808093 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:21.808374 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:21.808769 (+ 395us) server_negotiation.cc:299] Negotiation successful
0928 16:40:21.808989 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:40:21.810050 25653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:21.906782 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 25530
I20260928 16:40:21.923774 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 25167
I20260928 16:40:21.925307 25231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:22.058424 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 25167
I20260928 16:40:22.081300 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45029
--webserver_interface=127.10.247.254
--webserver_port=40239
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:45029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:22.478803 25709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:22.479188 25709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:22.479321 25709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:22.487310 25709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:22.487473 25709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:22.487525 25709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:22.487609 25709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:22.487670 25709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:22.498935 25709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:45029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=40239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:22.500877 25709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:22.502595 25709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:22.529240 25718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:22.529240 25716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:22.529301 25715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:22.530591 25709 server_base.cc:1034] running on GCE node
I20260928 16:40:22.531888 25709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:22.534026 25709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:22.535279 25709 hybrid_clock.cc:648] HybridClock initialized: now 1790613622535221 us; error 53 us; skew 500 ppm
I20260928 16:40:22.535766 25709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:22 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613622, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:22.539987 25709 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:40:22.542116 25709 webserver.cc:460] Webserver started at http://127.10.247.254:40239/ using document root <none> and password file <none>
I20260928 16:40:22.543056 25709 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:22.543216 25709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:22.550822 25709 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:40:22.554514 25725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:22.556351 25709 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260928 16:40:22.556561 25709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "416a40b028794fcdabeff6550154ebed"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "f5b761d47e1606a8e9e9da76c0bd4652"
server_key_iv: "50fcaa86c5e860cebc4680facab09d38"
server_key_version: "encryptionkey@0"
I20260928 16:40:22.557952 25709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:22.589133 25709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:22.594662 25709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:22.595002 25709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:22.616961 25709 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:45029
I20260928 16:40:22.616992 25776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:45029 every 8 connection(s)
I20260928 16:40:22.618942 25709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:40:22.623550 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 25709
I20260928 16:40:22.624758 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:37211
--local_ip_for_outbound_sockets=127.10.247.193
--tserver_master_addrs=127.10.247.254:45029
--webserver_port=36287
--webserver_interface=127.10.247.193
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:40:22.630004 25777 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:40:22.635619 25777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Bootstrap starting.
I20260928 16:40:22.660645 25777 log.cc:824] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:22.675661 25777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:40:22.676329 25777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Bootstrap complete.
I20260928 16:40:22.687194 25777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:22.688268 25777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 416a40b028794fcdabeff6550154ebed, State: Initialized, Role: FOLLOWER
I20260928 16:40:22.689145 25777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:22.689410 25777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:40:22.689549 25777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:40:22.689735 25777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 1 FOLLOWER]: Advancing to term 2
I20260928 16:40:22.694579 25777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:22.695266 25777 leader_election.cc:302] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 416a40b028794fcdabeff6550154ebed; no voters:
I20260928 16:40:22.696045 25777 leader_election.cc:288] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 16:40:22.696249 25781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 2 FOLLOWER]: Leader election won for term 2
I20260928 16:40:22.697453 25781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 2 LEADER]: Becoming Leader. State: Replica: 416a40b028794fcdabeff6550154ebed, State: Running, Role: LEADER
I20260928 16:40:22.698210 25781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:22.698815 25777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:40:22.701512 25782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "416a40b028794fcdabeff6550154ebed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } } }
I20260928 16:40:22.701982 25782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:22.702513 25783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 416a40b028794fcdabeff6550154ebed. Latest consensus state: current_term: 2 leader_uuid: "416a40b028794fcdabeff6550154ebed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } } }
I20260928 16:40:22.702836 25783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:22.714183 25787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:40:22.723944 25787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:40:22.725818 25787 catalog_manager.cc:1259] Loaded cluster ID: 73715f95e3fc44b28e4302e79ea8c648
I20260928 16:40:22.726038 25787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:40:22.729669 25787 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260928 16:40:22.746799 25787 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260928 16:40:22.757629 25787 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:40:22.759717 25787 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Loaded TSK: 0
I20260928 16:40:22.761518 25787 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 16:40:23.047644 25779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:23.048013 25779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:23.048156 25779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:23.056366 25779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:23.056511 25779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:23.056723 25779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:40:23.068256 25779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:37211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=36287
--tserver_master_addrs=127.10.247.254:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:23.070115 25779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:23.072057 25779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:23.091836 25804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:23.092097 25805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:23.098332 25807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:23.098734 25779 server_base.cc:1034] running on GCE node
I20260928 16:40:23.099754 25779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:23.101152 25779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:23.102415 25779 hybrid_clock.cc:648] HybridClock initialized: now 1790613623102339 us; error 68 us; skew 500 ppm
I20260928 16:40:23.102813 25779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:23 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:23.106596 25779 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:40:23.108542 25779 webserver.cc:460] Webserver started at http://127.10.247.193:36287/ using document root <none> and password file <none>
I20260928 16:40:23.109494 25779 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:23.109647 25779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:23.116707 25779 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 16:40:23.120157 25814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:23.122094 25779 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:40:23.122318 25779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "1659addd8a214901b259e07421fc8581"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "4374d8add0dc0b7ca3a7376c1ea5c834"
server_key_iv: "ad32b90253cd7df16c8c1dd02168936b"
server_key_version: "encryptionkey@0"
I20260928 16:40:23.123576 25779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:23.149948 25779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:23.155016 25779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:23.155339 25779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:23.156594 25779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:23.158300 25779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:23.158411 25779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:23.158529 25779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:23.158610 25779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:23.193347 25779 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:37211
I20260928 16:40:23.193406 25927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:37211 every 8 connection(s)
I20260928 16:40:23.195129 25779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:40:23.203578 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 25779
I20260928 16:40:23.204512 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:46121
--local_ip_for_outbound_sockets=127.10.247.194
--tserver_master_addrs=127.10.247.254:45029
--webserver_port=37731
--webserver_interface=127.10.247.194
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:40:23 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:23.225780 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:23.199065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:52389)
0928 16:40:23.200047 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:23.200109 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:23.202180 (+ 2071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:23.204024 (+ 1844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:23.204032 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:23.204859 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:23.205526 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:23.205546 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:23.207877 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:23.207884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:23.210018 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:23.210037 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:23.211643 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:23.211797 (+ 154us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:23.212131 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:23.213100 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:23.213148 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:23.215811 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:23.220433 (+ 4622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:23.220442 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:23.220458 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:23.221046 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:23.221542 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:23.221546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:23.221549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:23.221761 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:23.222663 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:23.222674 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:23.223081 (+ 407us) client_negotiation.cc:769] Sending connection context
0928 16:40:23.223363 (+ 282us) client_negotiation.cc:240] Negotiation successful
0928 16:40:23.224082 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":358,"threads_started":1}
I20260928 16:40:23.225785 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:23.198738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:52389 (local address 127.10.247.254:45029)
0928 16:40:23.199509 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:23.199523 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:23.200287 (+ 764us) server_negotiation.cc:407] Connection header received
0928 16:40:23.202834 (+ 2547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:23.202855 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:23.203423 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:23.203888 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:23.205771 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:23.207681 (+ 1910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:23.210234 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:23.211365 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:23.211604 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:23.216197 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:23.216208 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:23.216229 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:23.216283 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:23.220236 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:23.221219 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:23.221228 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:23.221234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:23.221416 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:23.221897 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:23.221902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:23.221904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:23.222512 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:23.222756 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:23.223657 (+ 901us) server_negotiation.cc:299] Negotiation successful
0928 16:40:23.224110 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":322,"threads_started":1}
I20260928 16:40:23.229184 25928 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45029
I20260928 16:40:23.229652 25928 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:23.230819 25928 heartbeater.cc:511] Master 127.10.247.254:45029 requested a full tablet report, sending...
I20260928 16:40:23.233769 25742 ts_manager.cc:194] Registered new tserver with Master: 1659addd8a214901b259e07421fc8581 (127.10.247.193:37211)
I20260928 16:40:23.235690 25742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:52389
W20260928 16:40:23.598296 25932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:23.598666 25932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:23.598819 25932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:23.606773 25932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:23.606921 25932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:23.607085 25932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:40:23.619141 25932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:46121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=37731
--tserver_master_addrs=127.10.247.254:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:23.621177 25932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:23.623340 25932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:23.641268 25937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:23.641207 25940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:23.641204 25938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:23.642015 25932 server_base.cc:1034] running on GCE node
I20260928 16:40:23.642757 25932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:23.643862 25932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:23.645071 25932 hybrid_clock.cc:648] HybridClock initialized: now 1790613623645012 us; error 50 us; skew 500 ppm
I20260928 16:40:23.645476 25932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:23 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:23.648973 25932 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:40:23.650681 25932 webserver.cc:460] Webserver started at http://127.10.247.194:37731/ using document root <none> and password file <none>
I20260928 16:40:23.651527 25932 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:23.651682 25932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:23.658306 25932 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:40:23.661660 25947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:23.663365 25932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260928 16:40:23.663638 25932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "71fbefddafda49f18658dd24387d4987"
format_stamp: "Formatted at 2026-09-28 16:40:20 on dist-test-slave-9fn6"
server_key: "d5c7bd288213c4123f4cdd37bfe83623"
server_key_iv: "236e8e6d47ebebfe8a89e9004c8507e8"
server_key_version: "encryptionkey@0"
I20260928 16:40:23.664819 25932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:23.679044 25932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:23.683864 25932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:23.684145 25932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:23.685415 25932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:23.686901 25932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:23.687004 25932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:23.687088 25932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:23.687147 25932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:23.720546 25932 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:46121
I20260928 16:40:23.720623 26059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:46121 every 8 connection(s)
I20260928 16:40:23.722275 25932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:40:23.730463 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 25932
I20260928 16:40:23.731204 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:43495
--local_ip_for_outbound_sockets=127.10.247.195
--tserver_master_addrs=127.10.247.254:45029
--webserver_port=36707
--webserver_interface=127.10.247.195
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:40:23 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:23.745843 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:23.725359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:36445 (local address 127.10.247.254:45029)
0928 16:40:23.725606 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:23.725612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:23.726989 (+ 1377us) server_negotiation.cc:407] Connection header received
0928 16:40:23.729238 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:23.729243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:23.729374 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:23.729469 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:23.731404 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:23.732503 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:23.734254 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:23.734898 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:23.735043 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:23.739127 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:23.739134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:23.739139 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:23.739188 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:23.742219 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:23.743216 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:23.743220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:23.743223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:23.743370 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:23.743835 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:23.743838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:23.743840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:23.744147 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:23.744266 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:23.745243 (+ 977us) server_negotiation.cc:299] Negotiation successful
0928 16:40:23.745473 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260928 16:40:23.747390 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:23.725913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:36445)
0928 16:40:23.726762 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:23.726821 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:23.728889 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:23.729756 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:23.729767 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:23.730405 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:23.731168 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:23.731191 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:23.732725 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:23.732730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:23.734094 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:23.734109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:23.735135 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:23.735141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:23.735375 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:23.736139 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:23.736168 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:23.738851 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:23.742403 (+ 3552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:23.742416 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:23.742442 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:23.743100 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:23.743457 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:23.743462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:23.743465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:23.743746 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:23.744270 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:23.744279 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:23.744805 (+ 526us) client_negotiation.cc:769] Sending connection context
0928 16:40:23.745150 (+ 345us) client_negotiation.cc:240] Negotiation successful
0928 16:40:23.745797 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":330,"threads_started":1}
I20260928 16:40:23.749146 26060 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45029
I20260928 16:40:23.749635 26060 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:23.750741 26060 heartbeater.cc:511] Master 127.10.247.254:45029 requested a full tablet report, sending...
I20260928 16:40:23.753245 25742 ts_manager.cc:194] Registered new tserver with Master: 71fbefddafda49f18658dd24387d4987 (127.10.247.194:46121)
I20260928 16:40:23.754191 25742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:36445
W20260928 16:40:24.120296 26063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:24.120734 26063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:24.120893 26063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:24.129364 26063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:24.129528 26063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:24.129702 26063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:40:24.140784 26063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:43495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=36707
--tserver_master_addrs=127.10.247.254:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:24.142771 26063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:24.144455 26063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:24.159940 26068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:24.159974 26069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:24.159951 26071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:24.162298 26063 server_base.cc:1034] running on GCE node
I20260928 16:40:24.163080 26063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:24.164304 26063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:24.165529 26063 hybrid_clock.cc:648] HybridClock initialized: now 1790613624165460 us; error 60 us; skew 500 ppm
I20260928 16:40:24.165899 26063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:24 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613624, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:24.169708 26063 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:40:24.171602 26063 webserver.cc:460] Webserver started at http://127.10.247.195:36707/ using document root <none> and password file <none>
I20260928 16:40:24.172477 26063 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:24.172644 26063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:24.179618 26063 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260928 16:40:24.182991 26078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:24.184729 26063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:40:24.184906 26063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "45b25079458041dfb0590947818682a1"
format_stamp: "Formatted at 2026-09-28 16:40:21 on dist-test-slave-9fn6"
server_key: "eccd213a67c76a8767fcdf0ec20a2bc5"
server_key_iv: "d655050514553e54f546b5cb23e81244"
server_key_version: "encryptionkey@0"
I20260928 16:40:24.186064 26063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:24.229372 26063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:24.234457 26063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:24.234768 26063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:24.236049 26063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:24.237715 26063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:24.237824 26063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:24.237939 26063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:24.238010 26063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:24.239761 25928 heartbeater.cc:503] Master 127.10.247.254:45029 was elected leader, sending a full tablet report...
I20260928 16:40:24.273355 26063 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:43495
I20260928 16:40:24.273438 26190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:43495 every 8 connection(s)
I20260928 16:40:24.275210 26063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:40:24.277100 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 26063
Sep 28 16:40:24 dist-test-slave-9fn6 krb5kdc[25154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613624, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:24.309592 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.278331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55696 (local address 127.10.247.254:45029)
0928 16:40:24.294168 (+ 15837us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.294177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.294201 (+ 24us) server_negotiation.cc:407] Connection header received
0928 16:40:24.294280 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.294286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.294377 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.294483 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:24.296672 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.297507 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.299543 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.300002 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.300107 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.303119 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:24.303124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:24.303128 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:24.303166 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:24.306324 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:24.307265 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:24.307270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:24.307272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:24.307442 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:24.307915 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:24.307921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:24.307924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:24.308264 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:24.308496 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.308899 (+ 403us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.309168 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15684,"thread_start_us":15802,"threads_started":1}
I20260928 16:40:24.312739 26194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.278331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:59229 (local address 127.10.247.254:45029)
0928 16:40:24.295583 (+ 17252us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.295590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.295609 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:40:24.295664 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.295669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.295739 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.295853 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:24.297860 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.299111 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.301025 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.301478 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.301591 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.305604 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:24.305609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:24.305612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:24.305665 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:24.309052 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:24.310057 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:24.310062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:24.310064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:24.310189 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:24.310715 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:24.310720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:24.310723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:24.311028 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:24.311176 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.312072 (+ 896us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.312293 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":17120,"thread_start_us":16929,"threads_started":1}
I20260928 16:40:24.313798 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.279158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:59229)
0928 16:40:24.289428 (+ 10270us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.289504 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:24.292545 (+ 3041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:24.296237 (+ 3692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:24.296249 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:24.296944 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:24.297677 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.297709 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.299310 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.299314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.300874 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.300888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.301724 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.301732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.301995 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.302668 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:24.302693 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:24.305350 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:24.309220 (+ 3870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:24.309230 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:24.309252 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:24.309891 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:24.310331 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:24.310337 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:24.310341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:24.310607 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:24.311176 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:24.311186 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:24.311667 (+ 481us) client_negotiation.cc:769] Sending connection context
0928 16:40:24.311920 (+ 253us) client_negotiation.cc:240] Negotiation successful
0928 16:40:24.312443 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9797,"thread_start_us":13318,"threads_started":1}
I20260928 16:40:24.315130 26191 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45029
I20260928 16:40:24.315457 26191 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:24.316313 26191 heartbeater.cc:511] Master 127.10.247.254:45029 requested a full tablet report, sending...
I20260928 16:40:24.317958 25742 ts_manager.cc:194] Registered new tserver with Master: 45b25079458041dfb0590947818682a1 (127.10.247.195:43495)
I20260928 16:40:24.318908 25742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:59229
I20260928 16:40:24.325551 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:40:24.333665 26194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.326655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.10.247.254:45029)
0928 16:40:24.326902 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.326909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.327508 (+ 599us) server_negotiation.cc:407] Connection header received
0928 16:40:24.327697 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.327701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.327874 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.328121 (+ 247us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:40:24.328534 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.329455 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.330986 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.331473 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.331582 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.332262 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:40:24.333021 (+ 759us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:40:24.333147 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.333264 (+ 117us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.333356 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:40:24.334754 25742 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:40:24.343158 25742 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:40:24.371693 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.358195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:35944)
0928 16:40:24.359920 (+ 1725us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.359966 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:24.360190 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:24.361602 (+ 1412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:24.361613 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:24.361652 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:24.362244 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.362266 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.364643 (+ 2377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.364650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.367380 (+ 2730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.367403 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.370353 (+ 2950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.370360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.370654 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.370718 (+ 64us) client_negotiation.cc:769] Sending connection context
0928 16:40:24.370891 (+ 173us) client_negotiation.cc:240] Negotiation successful
0928 16:40:24.371143 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1435,"thread_start_us":1243,"threads_started":1}
I20260928 16:40:24.373067 26202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.358231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35944 (local address 127.10.247.193:37211)
0928 16:40:24.359192 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.359206 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.360459 (+ 1253us) server_negotiation.cc:407] Connection header received
0928 16:40:24.360650 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.360661 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.361000 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.361237 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:24.362545 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.364381 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.367657 (+ 3276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.369663 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.369913 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.370166 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.372371 (+ 2205us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.372648 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":444,"threads_started":1}
I20260928 16:40:24.379040 25862 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4867445e4d4fb2b114e978db72b489 (DEFAULT_TABLE table=test-table [id=7f6f12175eaf4b2390e8ebb78a6b4dfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:24.381266 25862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4867445e4d4fb2b114e978db72b489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:24.383435 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.358670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:46121 (local address 127.0.0.1:37806)
0928 16:40:24.372007 (+ 13337us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.372029 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:24.372154 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:24.375742 (+ 3588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:24.375747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:24.375774 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:24.375993 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.376006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.378416 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.378422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.380634 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.380665 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.382514 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.382520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.382724 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.382761 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 16:40:24.382822 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 16:40:24.382987 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13221,"mutex_wait_us":24,"thread_start_us":16974,"threads_started":1}
I20260928 16:40:24.383472 26206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.361019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52722 (local address 127.10.247.195:43495)
0928 16:40:24.374754 (+ 13735us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.374764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.374800 (+ 36us) server_negotiation.cc:407] Connection header received
0928 16:40:24.374909 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.374917 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.375211 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.375448 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:24.376513 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.378250 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.380669 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.382228 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.382447 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.382714 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.382945 (+ 231us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.383128 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13509,"thread_start_us":14818,"threads_started":1}
I20260928 16:40:24.383497 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.361495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:43495 (local address 127.0.0.1:52722)
0928 16:40:24.373927 (+ 12432us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.373947 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:24.374081 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:24.375811 (+ 1730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:24.375816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:24.375860 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:24.376305 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.376320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.378421 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.378424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.380478 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.380493 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.382515 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.382528 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.382730 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.382770 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 16:40:24.382872 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 16:40:24.383019 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12294,"mutex_wait_us":43,"spinlock_wait_cycles":2944,"thread_start_us":12115,"threads_started":1}
I20260928 16:40:24.384704 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.359285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37806 (local address 127.10.247.194:46121)
0928 16:40:24.374636 (+ 15351us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.374647 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.374684 (+ 37us) server_negotiation.cc:407] Connection header received
0928 16:40:24.374830 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.374842 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.375208 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.376204 (+ 996us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:24.376417 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.378222 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.380843 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.382298 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.383860 (+ 1562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.384186 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.384244 (+ 58us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.384389 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15124,"thread_start_us":16105,"threads_started":1}
I20260928 16:40:24.388729 26126 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4867445e4d4fb2b114e978db72b489 (DEFAULT_TABLE table=test-table [id=7f6f12175eaf4b2390e8ebb78a6b4dfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:24.390712 26126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4867445e4d4fb2b114e978db72b489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:24.392324 25995 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4867445e4d4fb2b114e978db72b489 (DEFAULT_TABLE table=test-table [id=7f6f12175eaf4b2390e8ebb78a6b4dfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:24.393678 25995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4867445e4d4fb2b114e978db72b489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:24.400650 26208 tablet_bootstrap.cc:492] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Bootstrap starting.
I20260928 16:40:24.405058 26208 tablet_bootstrap.cc:654] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:24.406575 26208 log.cc:824] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:24.407613 26209 tablet_bootstrap.cc:492] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: Bootstrap starting.
I20260928 16:40:24.410619 26211 tablet_bootstrap.cc:492] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: Bootstrap starting.
I20260928 16:40:24.411743 26209 tablet_bootstrap.cc:654] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:24.412163 26208 tablet_bootstrap.cc:492] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: No bootstrap required, opened a new log
I20260928 16:40:24.412523 26208 ts_tablet_manager.cc:1397] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260928 16:40:24.412963 26209 log.cc:824] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:24.414459 26211 tablet_bootstrap.cc:654] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:24.415792 26209 tablet_bootstrap.cc:492] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: No bootstrap required, opened a new log
I20260928 16:40:24.415768 26211 log.cc:824] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:24.416060 26209 ts_tablet_manager.cc:1397] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 16:40:24.419117 26211 tablet_bootstrap.cc:492] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: No bootstrap required, opened a new log
I20260928 16:40:24.419405 26211 ts_tablet_manager.cc:1397] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260928 16:40:24.422147 26208 raft_consensus.cc:348] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:24.422653 26208 raft_consensus.cc:374] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:24.422756 26208 raft_consensus.cc:729] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1659addd8a214901b259e07421fc8581, State: Initialized, Role: FOLLOWER
I20260928 16:40:24.423504 26208 consensus_queue.cc:260] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:24.425529 26208 ts_tablet_manager.cc:1428] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260928 16:40:24.427059 26209 raft_consensus.cc:348] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:24.427644 26209 raft_consensus.cc:374] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:24.427748 26209 raft_consensus.cc:729] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b25079458041dfb0590947818682a1, State: Initialized, Role: FOLLOWER
I20260928 16:40:24.428423 26209 consensus_queue.cc:260] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:24.430298 26191 heartbeater.cc:503] Master 127.10.247.254:45029 was elected leader, sending a full tablet report...
I20260928 16:40:24.430346 26211 raft_consensus.cc:348] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:24.430807 26209 ts_tablet_manager.cc:1428] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260928 16:40:24.430847 26211 raft_consensus.cc:374] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:24.430936 26211 raft_consensus.cc:729] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71fbefddafda49f18658dd24387d4987, State: Initialized, Role: FOLLOWER
I20260928 16:40:24.431597 26211 consensus_queue.cc:260] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:24.433782 26060 heartbeater.cc:503] Master 127.10.247.254:45029 was elected leader, sending a full tablet report...
I20260928 16:40:24.434398 26211 ts_tablet_manager.cc:1428] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: Time spent starting tablet: real 0.015s user 0.007s sys 0.009s
W20260928 16:40:24.447909 25929 tablet.cc:2369] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:40:24.474572 26061 tablet.cc:2369] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:40:24.527235 26192 tablet.cc:2369] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:40:24.544566 26216 raft_consensus.cc:482] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:40:24.545019 26216 raft_consensus.cc:504] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:24.547250 26216 leader_election.cc:288] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1659addd8a214901b259e07421fc8581 (127.10.247.193:37211), 45b25079458041dfb0590947818682a1 (127.10.247.195:43495)
I20260928 16:40:24.556070 26202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.547837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:34527 (local address 127.10.247.193:37211)
0928 16:40:24.548117 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.548125 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.548649 (+ 524us) server_negotiation.cc:407] Connection header received
0928 16:40:24.548983 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.548990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.549085 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.549158 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:24.550054 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.551237 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.554149 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.555012 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.555213 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.555316 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.555627 (+ 311us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.555785 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 16:40:24.556150 26206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.548830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:35463 (local address 127.10.247.195:43495)
0928 16:40:24.549074 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.549080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.549377 (+ 297us) server_negotiation.cc:407] Connection header received
0928 16:40:24.549518 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.549523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.549615 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.549718 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:24.550278 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.551412 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.554112 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.555141 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.555340 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.555425 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.555732 (+ 307us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.555880 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:40:24.556221 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.547746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.10.247.194:34527)
0928 16:40:24.548488 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.548510 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:24.548754 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:24.549367 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:24.549372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:24.549401 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:24.549852 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.549871 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.551357 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.551360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.553922 (+ 2562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.553939 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.555270 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.555277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.555446 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.555478 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 16:40:24.555575 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 16:40:24.555732 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":33,"thread_start_us":404,"threads_started":1}
I20260928 16:40:24.556818 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.548574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:43495 (local address 127.10.247.194:35463)
0928 16:40:24.549257 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.549271 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:24.549392 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:24.549732 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:24.549738 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:24.549756 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:24.550115 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.550124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.551576 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.551581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.553923 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:24.553939 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.555377 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.555384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:24.555574 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.555598 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 16:40:24.555680 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 16:40:24.555814 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":71,"thread_start_us":324,"threads_started":1}
I20260928 16:40:24.557559 25882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4867445e4d4fb2b114e978db72b489" candidate_uuid: "71fbefddafda49f18658dd24387d4987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1659addd8a214901b259e07421fc8581" is_pre_election: true
I20260928 16:40:24.557838 26146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4867445e4d4fb2b114e978db72b489" candidate_uuid: "71fbefddafda49f18658dd24387d4987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b25079458041dfb0590947818682a1" is_pre_election: true
I20260928 16:40:24.558349 25882 raft_consensus.cc:2454] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71fbefddafda49f18658dd24387d4987 in term 0.
I20260928 16:40:24.558409 26146 raft_consensus.cc:2454] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71fbefddafda49f18658dd24387d4987 in term 0.
I20260928 16:40:24.559173 25951 leader_election.cc:302] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1659addd8a214901b259e07421fc8581, 71fbefddafda49f18658dd24387d4987; no voters:
I20260928 16:40:24.559603 26216 raft_consensus.cc:2781] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:40:24.559762 26216 raft_consensus.cc:482] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:40:24.559880 26216 raft_consensus.cc:3037] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:24.562673 26216 raft_consensus.cc:504] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:24.563354 26216 leader_election.cc:288] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [CANDIDATE]: Term 1 election: Requested vote from peers 1659addd8a214901b259e07421fc8581 (127.10.247.193:37211), 45b25079458041dfb0590947818682a1 (127.10.247.195:43495)
I20260928 16:40:24.563827 25882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4867445e4d4fb2b114e978db72b489" candidate_uuid: "71fbefddafda49f18658dd24387d4987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1659addd8a214901b259e07421fc8581"
I20260928 16:40:24.563935 26146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4867445e4d4fb2b114e978db72b489" candidate_uuid: "71fbefddafda49f18658dd24387d4987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b25079458041dfb0590947818682a1"
I20260928 16:40:24.564040 25882 raft_consensus.cc:3037] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:24.564160 26146 raft_consensus.cc:3037] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:24.566570 25882 raft_consensus.cc:2454] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71fbefddafda49f18658dd24387d4987 in term 1.
I20260928 16:40:24.566636 26146 raft_consensus.cc:2454] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71fbefddafda49f18658dd24387d4987 in term 1.
I20260928 16:40:24.566968 25951 leader_election.cc:302] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1659addd8a214901b259e07421fc8581, 71fbefddafda49f18658dd24387d4987; no voters:
I20260928 16:40:24.567296 26216 raft_consensus.cc:2781] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:40:24.567777 26216 raft_consensus.cc:686] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 1 LEADER]: Becoming Leader. State: Replica: 71fbefddafda49f18658dd24387d4987, State: Running, Role: LEADER
I20260928 16:40:24.568349 26216 consensus_queue.cc:237] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } }
I20260928 16:40:24.573711 25741 catalog_manager.cc:5495] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71fbefddafda49f18658dd24387d4987 (127.10.247.194). New cstate: current_term: 1 leader_uuid: "71fbefddafda49f18658dd24387d4987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71fbefddafda49f18658dd24387d4987" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 46121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260928 16:40:24.624337 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.618194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.10.247.194:46121)
0928 16:40:24.618436 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.618442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.618465 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:40:24.618630 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.618635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.618732 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.618876 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:40:24.619324 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.620125 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.621775 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.622222 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.622338 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.622927 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:40:24.623639 (+ 712us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:40:24.623796 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.623901 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.624016 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 16:40:24.635654 26146 raft_consensus.cc:1261] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [term 1 FOLLOWER]: Refusing update from remote peer 71fbefddafda49f18658dd24387d4987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:24.635654 25882 raft_consensus.cc:1261] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [term 1 FOLLOWER]: Refusing update from remote peer 71fbefddafda49f18658dd24387d4987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:24.636823 26219 consensus_queue.cc:1035] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45b25079458041dfb0590947818682a1" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:40:24.637301 26216 consensus_queue.cc:1035] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1659addd8a214901b259e07421fc8581" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:40:24.650329 26224 mvcc.cc:205] Tried to move back new op lower bound from 7334353406493380608 to 7334353406236483584. Current Snapshot: MvccSnapshot[applied={T|T < 7334353406493380608}]
I20260928 16:40:24.669339 25742 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:40:24.669734 25742 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:40:24.673991 25742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Sending DeleteTablet for 3 replicas of tablet 8a4867445e4d4fb2b114e978db72b489
I20260928 16:40:24.675663 26126 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4867445e4d4fb2b114e978db72b489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:24 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:52722
I20260928 16:40:24.675686 25862 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4867445e4d4fb2b114e978db72b489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:24 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:35944
I20260928 16:40:24.675871 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 25779
I20260928 16:40:24.675827 25995 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4867445e4d4fb2b114e978db72b489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:24 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:37806
I20260928 16:40:24.685200 26233 tablet_replica.cc:330] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: stopping tablet replica
I20260928 16:40:24.685858 26233 raft_consensus.cc:2229] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:40:24.686867 26233 raft_consensus.cc:2258] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:24.689805 26232 tablet_replica.cc:330] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: stopping tablet replica
I20260928 16:40:24.690605 26232 raft_consensus.cc:2229] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:40:24.691514 26232 raft_consensus.cc:2258] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:24.692540 26231 tablet_replica.cc:330] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: stopping tablet replica
I20260928 16:40:24.692999 26231 raft_consensus.cc:2229] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:40:24.693490 26231 raft_consensus.cc:2258] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:24.695667 26231 ts_tablet_manager.cc:1905] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:24.697480 26232 ts_tablet_manager.cc:1905] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:24.700417 26202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.676984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.10.247.193:37211)
0928 16:40:24.677682 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.677688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.677708 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:40:24.677776 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.677782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.677881 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.677987 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:24.680185 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.681530 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.683619 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.684376 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.684514 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.689151 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:24.689178 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:24.689203 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:24.689261 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:24.694353 (+ 5092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:24.695461 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:24.695471 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:24.695474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:24.695696 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:24.697157 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:24.697163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:24.697166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:24.697924 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:24.698282 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.699721 (+ 1439us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.699982 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551}
I20260928 16:40:24.702646 26233 ts_tablet_manager.cc:1905] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:24.703866 25923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:24.709492 26232 ts_tablet_manager.cc:1918] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:24.709715 26232 log.cc:1196] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/8a4867445e4d4fb2b114e978db72b489
I20260928 16:40:24.710335 26232 ts_tablet_manager.cc:1939] T 8a4867445e4d4fb2b114e978db72b489 P 45b25079458041dfb0590947818682a1: Deleting consensus metadata
I20260928 16:40:24.712690 26233 ts_tablet_manager.cc:1918] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:24.712873 25729 catalog_manager.cc:4841] TS 45b25079458041dfb0590947818682a1 (127.10.247.195:43495): tablet 8a4867445e4d4fb2b114e978db72b489 (table test-table [id=7f6f12175eaf4b2390e8ebb78a6b4dfc]) successfully deleted
I20260928 16:40:24.713022 26233 log.cc:1196] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/8a4867445e4d4fb2b114e978db72b489
I20260928 16:40:24.713825 26233 ts_tablet_manager.cc:1939] T 8a4867445e4d4fb2b114e978db72b489 P 71fbefddafda49f18658dd24387d4987: Deleting consensus metadata
I20260928 16:40:24.716089 25726 catalog_manager.cc:4841] TS 71fbefddafda49f18658dd24387d4987 (127.10.247.194:46121): tablet 8a4867445e4d4fb2b114e978db72b489 (table test-table [id=7f6f12175eaf4b2390e8ebb78a6b4dfc]) successfully deleted
I20260928 16:40:24.820416 26231 ts_tablet_manager.cc:1918] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:24.820762 26231 log.cc:1196] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/8a4867445e4d4fb2b114e978db72b489
I20260928 16:40:24.821388 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 25779
I20260928 16:40:24.821419 26231 ts_tablet_manager.cc:1939] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Deleting consensus metadata
I20260928 16:40:24.848197 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 25932
W20260928 16:40:24.848397 25729 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.247.193:37211 (error 108)
I20260928 16:40:24.850883 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.850296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:35962)
0928 16:40:24.850509 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.850649 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260928 16:40:24.851231 25729 catalog_manager.cc:4584] TS 1659addd8a214901b259e07421fc8581 (127.10.247.193:37211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a4867445e4d4fb2b114e978db72b489: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37211: connect: Connection refused (error 111)
I20260928 16:40:24.865059 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.849208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.10.247.194:46121)
0928 16:40:24.849443 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:24.849451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:24.849796 (+ 345us) server_negotiation.cc:407] Connection header received
0928 16:40:24.849891 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:24.849897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:24.850029 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:24.850144 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:24.852083 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.853155 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.854528 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:24.854983 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:24.855097 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:24.858444 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:24.858462 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:24.858481 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:24.858526 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:24.861851 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:24.862639 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:24.862649 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:24.862651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:24.862822 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:24.863210 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:24.863215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:24.863217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:24.863680 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:24.864049 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:24.864350 (+ 301us) server_negotiation.cc:299] Negotiation successful
0928 16:40:24.864597 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260928 16:40:24.865936 26055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:24.905508 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.904886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:35972)
0928 16:40:24.905139 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.905281 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260928 16:40:24.906734 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.906309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:35986)
0928 16:40:24.906495 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.906595 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 16:40:24.967840 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.967197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:35994)
0928 16:40:24.967484 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.967608 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260928 16:40:24.969085 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:24.968707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:36010)
0928 16:40:24.968848 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:24.968933 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 16:40:24.995385 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 25932
I20260928 16:40:25.014743 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 26063
I20260928 16:40:25.029481 26206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:25.015641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.10.247.195:43495)
0928 16:40:25.015831 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:25.015836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:25.015858 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:40:25.016022 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:25.016026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:25.016116 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:25.016212 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:25.017866 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:25.018683 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:25.020124 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:25.020570 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:25.020742 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:25.023604 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:25.023624 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:25.023645 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:25.023694 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:25.026642 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:25.027391 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:25.027400 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:25.027403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:25.027573 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:25.027863 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:25.027866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:25.027868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:25.028275 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:25.028542 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:25.028953 (+ 411us) server_negotiation.cc:299] Negotiation successful
0928 16:40:25.029141 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:40:25.030160 26186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:25.050065 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:25.049415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:36020)
0928 16:40:25.049685 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:25.049809 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260928 16:40:25.051311 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:25.050867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:36022)
0928 16:40:25.051096 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:25.051178 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260928 16:40:25.127952 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 26063
I20260928 16:40:25.145323 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 25709
I20260928 16:40:25.146725 25772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:25.275228 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:25.274536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37211 (local address 127.0.0.1:36036)
0928 16:40:25.274856 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:25.274979 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260928 16:40:25.275449 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 25709
I20260928 16:40:25.298488 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45029
--webserver_interface=127.10.247.254
--webserver_port=40239
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:45029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:25.672796 26239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:25.673194 26239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:25.673343 26239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:25.681466 26239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:25.681603 26239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:25.681680 26239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:25.681764 26239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:25.681819 26239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:25.692771 26239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:45029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=40239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:25.694571 26239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:25.696249 26239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:25.709246 26244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:25.709367 26245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:25.709396 26247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:25.710429 26239 server_base.cc:1034] running on GCE node
I20260928 16:40:25.711421 26239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:25.713253 26239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:25.714471 26239 hybrid_clock.cc:648] HybridClock initialized: now 1790613625714409 us; error 50 us; skew 500 ppm
I20260928 16:40:25.714879 26239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:25 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613625, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:25.724269 26239 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:40:25.726358 26239 webserver.cc:460] Webserver started at http://127.10.247.254:40239/ using document root <none> and password file <none>
I20260928 16:40:25.727280 26239 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:25.727418 26239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:25.734251 26239 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260928 16:40:25.739593 26254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:25.741236 26239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260928 16:40:25.741420 26239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "416a40b028794fcdabeff6550154ebed"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "f5b761d47e1606a8e9e9da76c0bd4652"
server_key_iv: "50fcaa86c5e860cebc4680facab09d38"
server_key_version: "encryptionkey@0"
I20260928 16:40:25.742535 26239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:25.760839 26239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:25.766100 26239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:25.766394 26239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:25.786306 26239 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:45029
I20260928 16:40:25.786338 26305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:45029 every 8 connection(s)
I20260928 16:40:25.788060 26239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:40:25.796377 26306 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:40:25.797292 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 26239
I20260928 16:40:25.798346 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:37211
--local_ip_for_outbound_sockets=127.10.247.193
--tserver_master_addrs=127.10.247.254:45029
--webserver_port=36287
--webserver_interface=127.10.247.193
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:40:25.800702 26306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Bootstrap starting.
I20260928 16:40:25.825465 26306 log.cc:824] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:25.844925 26306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:40:25.845559 26306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed: Bootstrap complete.
I20260928 16:40:25.855808 26306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:25.856899 26306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 416a40b028794fcdabeff6550154ebed, State: Initialized, Role: FOLLOWER
I20260928 16:40:25.857618 26306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:25.857867 26306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:40:25.857991 26306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:40:25.858177 26306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 2 FOLLOWER]: Advancing to term 3
I20260928 16:40:25.862537 26306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:25.863116 26306 leader_election.cc:302] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 416a40b028794fcdabeff6550154ebed; no voters:
I20260928 16:40:25.863826 26306 leader_election.cc:288] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 16:40:25.864002 26310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 3 FOLLOWER]: Leader election won for term 3
I20260928 16:40:25.865149 26310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [term 3 LEADER]: Becoming Leader. State: Replica: 416a40b028794fcdabeff6550154ebed, State: Running, Role: LEADER
I20260928 16:40:25.865859 26310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } }
I20260928 16:40:25.866338 26306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:40:25.868873 26312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 416a40b028794fcdabeff6550154ebed. Latest consensus state: current_term: 3 leader_uuid: "416a40b028794fcdabeff6550154ebed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } } }
I20260928 16:40:25.869601 26312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:25.869609 26311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "416a40b028794fcdabeff6550154ebed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416a40b028794fcdabeff6550154ebed" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45029 } } }
I20260928 16:40:25.872112 26311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 416a40b028794fcdabeff6550154ebed [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:25.878225 26316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:40:25.889364 26316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:40:25.891404 26316 catalog_manager.cc:1259] Loaded cluster ID: 73715f95e3fc44b28e4302e79ea8c648
I20260928 16:40:25.891610 26316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260928 16:40:25.893932 26316 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd1d7dfd9ad google::LogMessage::Fail() at ??:0
@ 0x7fd1d7e0088a google::LogMessage::SendToLog() at ??:0
@ 0x7fd1d7dfd4de google::LogMessage::Flush() at ??:0
@ 0x7fd1d7dfea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd1ed830512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd1ed82d976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd1ed8b5610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd1ed8b5439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd1daa49345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd1daa4c91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd1daa4c799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd1daa27eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd1ddd016db start_thread at ??:0
@ 0x7fd1d6c9761f clone at ??:0
W20260928 16:40:26.290261 26308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:26.290637 26308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:26.290745 26308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:26.298753 26308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:26.298904 26308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:26.299050 26308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:40:26.310232 26308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:37211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=36287
--tserver_master_addrs=127.10.247.254:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:26.311977 26308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:26.313699 26308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:26.329273 26348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:26.329294 26347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:26.329360 26350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:26.330406 26308 server_base.cc:1034] running on GCE node
I20260928 16:40:26.331138 26308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:26.332319 26308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:26.333555 26308 hybrid_clock.cc:648] HybridClock initialized: now 1790613626333446 us; error 67 us; skew 500 ppm
I20260928 16:40:26.333923 26308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:26 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613626, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:26.337509 26308 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:40:26.339287 26308 webserver.cc:460] Webserver started at http://127.10.247.193:36287/ using document root <none> and password file <none>
I20260928 16:40:26.340126 26308 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:26.340242 26308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:26.346872 26308 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:40:26.350226 26357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:26.351863 26308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:40:26.352051 26308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "1659addd8a214901b259e07421fc8581"
format_stamp: "Formatted at 2026-09-28 16:40:19 on dist-test-slave-9fn6"
server_key: "4374d8add0dc0b7ca3a7376c1ea5c834"
server_key_iv: "ad32b90253cd7df16c8c1dd02168936b"
server_key_version: "encryptionkey@0"
I20260928 16:40:26.353273 26308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:26.376003 26308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:26.381264 26308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:26.381573 26308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:26.382824 26308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:26.384979 26364 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260928 16:40:26.391479 26364 ts_tablet_manager.cc:1852] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260928 16:40:26.392599 26364 ts_tablet_manager.cc:1905] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:26.401031 26364 ts_tablet_manager.cc:1918] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:26.401335 26364 ts_tablet_manager.cc:1939] T 8a4867445e4d4fb2b114e978db72b489 P 1659addd8a214901b259e07421fc8581: Deleting consensus metadata
I20260928 16:40:26.401811 26308 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260928 16:40:26.401969 26308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.000s sys 0.001s
I20260928 16:40:26.402143 26308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:26.402235 26308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:26.439445 26308 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:37211
I20260928 16:40:26.439482 26471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:37211 every 8 connection(s)
I20260928 16:40:26.441321 26308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:40:26.447422 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:26.445131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:46349)
0928 16:40:26.446018 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:26.446321 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":335,"threads_started":1}
W20260928 16:40:26.448123 26472 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:45029 (0 consecutive failures): Network error: Failed to ping master at 127.10.247.254:45029: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
I20260928 16:40:26.449168 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:26.448780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:36619)
0928 16:40:26.448947 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:26.449031 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 16:40:26.450070 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:26.449721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:38267)
0928 16:40:26.449867 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:26.449944 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260928 16:40:26.450292 26472 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 16:40:26.451398 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 26308
I20260928 16:40:26.452097 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:46121
--local_ip_for_outbound_sockets=127.10.247.194
--tserver_master_addrs=127.10.247.254:45029
--webserver_port=37731
--webserver_interface=127.10.247.194
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:26.848641 26475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:26.849081 26475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:26.849241 26475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:26.857548 26475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:26.857728 26475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:26.857908 26475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:40:26.869072 26475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:46121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=37731
--tserver_master_addrs=127.10.247.254:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:26.870878 26475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:26.872540 26475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:26.887509 26480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:26.887611 26483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:26.887542 26481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:26.888415 26475 server_base.cc:1034] running on GCE node
I20260928 16:40:26.889183 26475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:26.890323 26475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:26.891535 26475 hybrid_clock.cc:648] HybridClock initialized: now 1790613626891473 us; error 54 us; skew 500 ppm
I20260928 16:40:26.891897 26475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:26 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613626, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:26.895439 26475 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:40:26.897295 26475 webserver.cc:460] Webserver started at http://127.10.247.194:37731/ using document root <none> and password file <none>
I20260928 16:40:26.898125 26475 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:26.898265 26475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:26.905210 26475 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 16:40:26.908489 26490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:26.910261 26475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:40:26.910442 26475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "71fbefddafda49f18658dd24387d4987"
format_stamp: "Formatted at 2026-09-28 16:40:20 on dist-test-slave-9fn6"
server_key: "d5c7bd288213c4123f4cdd37bfe83623"
server_key_iv: "236e8e6d47ebebfe8a89e9004c8507e8"
server_key_version: "encryptionkey@0"
I20260928 16:40:26.911636 26475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:26.927004 26475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:26.931893 26475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:26.932189 26475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:26.933396 26475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:26.934981 26475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:26.935098 26475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:26.935257 26475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:26.935338 26475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:26.969133 26475 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:46121
I20260928 16:40:26.969192 26604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:46121 every 8 connection(s)
I20260928 16:40:26.970899 26475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:40:26.973429 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 26475
I20260928 16:40:26.974143 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:43495
--local_ip_for_outbound_sockets=127.10.247.195
--tserver_master_addrs=127.10.247.254:45029
--webserver_port=36707
--webserver_interface=127.10.247.195
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:42557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:40:26.976828 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:26.974421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:37593)
0928 16:40:26.975362 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:26.975653 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":351,"threads_started":1}
W20260928 16:40:26.977502 26605 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:45029 (0 consecutive failures): Network error: Failed to ping master at 127.10.247.254:45029: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
I20260928 16:40:26.978494 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:26.978137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:48415)
0928 16:40:26.978290 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:26.978367 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 16:40:26.979369 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:26.979064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:54335)
0928 16:40:26.979180 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:26.979254 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 16:40:26.979609 26605 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 16:40:27.352865 26608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:27.353228 26608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:27.353363 26608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:27.361382 26608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:27.361531 26608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:27.361707 26608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:40:27.372838 26608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:42557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:43495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=36707
--tserver_master_addrs=127.10.247.254:45029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:27.374686 26608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:27.376362 26608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:27.391752 26613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:27.391695 26616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:27.391709 26614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:27.392927 26608 server_base.cc:1034] running on GCE node
I20260928 16:40:27.393692 26608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:27.394832 26608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:27.396097 26608 hybrid_clock.cc:648] HybridClock initialized: now 1790613627396033 us; error 52 us; skew 500 ppm
I20260928 16:40:27.396447 26608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:27 dist-test-slave-9fn6 krb5kdc[25154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613627, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:27.399915 26608 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:40:27.401695 26608 webserver.cc:460] Webserver started at http://127.10.247.195:36707/ using document root <none> and password file <none>
I20260928 16:40:27.402513 26608 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:27.402643 26608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:27.409067 26608 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:40:27.412237 26623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:27.413908 26608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:40:27.414115 26608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "45b25079458041dfb0590947818682a1"
format_stamp: "Formatted at 2026-09-28 16:40:21 on dist-test-slave-9fn6"
server_key: "eccd213a67c76a8767fcdf0ec20a2bc5"
server_key_iv: "d655050514553e54f546b5cb23e81244"
server_key_version: "encryptionkey@0"
I20260928 16:40:27.415278 26608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:27.441005 26608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:27.445938 26608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:27.446240 26608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:27.447362 26608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:27.448899 26608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:27.449023 26608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:27.449137 26608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:27.449219 26608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:27.452026 26631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:27.450973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:44185)
0928 16:40:27.451571 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:27.451704 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":244,"threads_started":1}
I20260928 16:40:27.483119 26608 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:43495
I20260928 16:40:27.483177 26736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:43495 every 8 connection(s)
I20260928 16:40:27.484886 26608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:40:27.490360 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:27.488183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:55371)
0928 16:40:27.489068 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:27.489333 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":352,"threads_started":1}
W20260928 16:40:27.491061 26737 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:45029 (0 consecutive failures): Network error: Failed to ping master at 127.10.247.254:45029: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
I20260928 16:40:27.491954 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:27.491610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:37305)
0928 16:40:27.491769 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:27.491842 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 16:40:27.492833 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:27.492475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:47829)
0928 16:40:27.492644 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:27.492722 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260928 16:40:27.493073 26737 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 16:40:27.494882 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 26608
I20260928 16:40:27.981362 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:27.980324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:48491)
0928 16:40:27.980891 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:27.981046 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":288,"threads_started":1}
I20260928 16:40:28.453980 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:28.453078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:35989)
0928 16:40:28.453639 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:28.453747 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":281,"threads_started":1}
I20260928 16:40:28.494688 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:28.493740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:49643)
0928 16:40:28.494284 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:28.494422 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":241,"threads_started":1}
I20260928 16:40:28.983366 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:28.982491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:37925)
0928 16:40:28.983029 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:28.983130 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":269,"threads_started":1}
I20260928 16:40:29.455926 26745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:29.455044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:45549)
0928 16:40:29.455606 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:29.455723 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":270,"threads_started":1}
I20260928 16:40:29.496971 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:29.495847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:60787)
0928 16:40:29.496542 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:29.496707 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":384,"threads_started":1}
I20260928 16:40:29.985452 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:29.984541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:39917)
0928 16:40:29.985109 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:29.985225 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":308,"threads_started":1}
I20260928 16:40:30.458061 26748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:30.457140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:48309)
0928 16:40:30.457730 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:30.457835 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":315,"threads_started":1}
I20260928 16:40:30.499099 26749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:30.498077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:34589)
0928 16:40:30.498707 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:30.498837 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":304,"threads_started":1}
I20260928 16:40:30.987458 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:30.986557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:56069)
0928 16:40:30.987099 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:30.987219 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":284,"threads_started":1}
I20260928 16:40:31.460446 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:31.459398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:35569)
0928 16:40:31.460124 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:31.460232 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":324,"threads_started":1}
I20260928 16:40:31.501199 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:31.500118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:47937)
0928 16:40:31.500830 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:31.500948 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":304,"threads_started":1}
I20260928 16:40:31.989669 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:31.988754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:38253)
0928 16:40:31.989298 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:31.989408 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":273,"threads_started":1}
I20260928 16:40:32.462740 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:32.461648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:54325)
0928 16:40:32.462305 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:32.462445 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":278,"threads_started":1}
I20260928 16:40:32.503309 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:32.502374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:39579)
0928 16:40:32.502965 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:32.503064 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":312,"threads_started":1}
I20260928 16:40:32.992142 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:32.991104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:42523)
0928 16:40:32.991750 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:32.991880 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":329,"threads_started":1}
I20260928 16:40:33.465018 26757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:33.463955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:54631)
0928 16:40:33.464596 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:33.464752 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":274,"threads_started":1}
I20260928 16:40:33.505470 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:33.504465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:47065)
0928 16:40:33.505145 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:33.505260 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":339,"threads_started":1}
I20260928 16:40:33.994481 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:33.993409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:42537)
0928 16:40:33.994079 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:33.994194 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":269,"threads_started":1}
I20260928 16:40:34.467002 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:34.466096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:54431)
0928 16:40:34.466636 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:34.466761 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":270,"threads_started":1}
I20260928 16:40:34.507774 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:34.506636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:54989)
0928 16:40:34.507370 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:34.507492 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":319,"threads_started":1}
I20260928 16:40:34.996737 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:34.995735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:57007)
0928 16:40:34.996321 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:34.996450 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":308,"threads_started":1}
I20260928 16:40:35.469257 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:35.468218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:41635)
0928 16:40:35.468907 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:35.469026 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":369,"threads_started":1}
I20260928 16:40:35.509907 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:35.508976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:60337)
0928 16:40:35.509566 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:35.509690 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":303,"threads_started":1}
I20260928 16:40:35.998807 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:35.997870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:55651)
0928 16:40:35.998470 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:35.998578 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":309,"threads_started":1}
I20260928 16:40:36.471367 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:36.470405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:51349)
0928 16:40:36.471013 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:36.471123 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":328,"threads_started":1}
I20260928 16:40:36.512144 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:36.511205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:51427)
0928 16:40:36.511813 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:36.511923 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":275,"threads_started":1}
I20260928 16:40:37.001118 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:37.000149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:40853)
0928 16:40:37.000753 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:37.000867 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":303,"threads_started":1}
I20260928 16:40:37.473556 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:37.472457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:41005)
0928 16:40:37.473204 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:37.473319 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":378,"threads_started":1}
I20260928 16:40:37.514525 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:37.513517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:47359)
0928 16:40:37.514186 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:37.514306 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":346,"threads_started":1}
I20260928 16:40:38.003403 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:38.002405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:48931)
0928 16:40:38.003014 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:38.003144 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":284,"threads_started":1}
I20260928 16:40:38.475728 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:38.474713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:54301)
0928 16:40:38.475335 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:38.475447 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":324,"threads_started":1}
I20260928 16:40:38.516538 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:38.515636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:56207)
0928 16:40:38.516190 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:38.516306 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":283,"threads_started":1}
I20260928 16:40:39.005882 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:39.004895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:33253)
0928 16:40:39.005536 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:39.005650 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":355,"threads_started":1}
I20260928 16:40:39.477770 26775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:39.476829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:53617)
0928 16:40:39.477417 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:39.477545 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":290,"threads_started":1}
I20260928 16:40:39.518697 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:39.517789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:44695)
0928 16:40:39.518373 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:39.518485 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":280,"threads_started":1}
I20260928 16:40:40.008371 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:40.007190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:47209)
0928 16:40:40.007870 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:40.008041 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":340,"threads_started":1}
I20260928 16:40:40.480108 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:40.479115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:60397)
0928 16:40:40.479733 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:40.479859 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":317,"threads_started":1}
I20260928 16:40:40.520848 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:40.519884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:35807)
0928 16:40:40.520448 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:40.520559 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":267,"threads_started":1}
I20260928 16:40:41.010524 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:41.009626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:49201)
0928 16:40:41.010175 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:41.010281 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":271,"threads_started":1}
I20260928 16:40:41.482571 26781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:41.481588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:35607)
0928 16:40:41.482206 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:41.482353 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":292,"threads_started":1}
I20260928 16:40:41.523018 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:41.522000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:51871)
0928 16:40:41.522619 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:41.522731 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":289,"threads_started":1}
I20260928 16:40:42.012995 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:42.012007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.194:35915)
0928 16:40:42.012583 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:42.012761 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":269,"threads_started":1}
I20260928 16:40:42.484655 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:42.483625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.193:40681)
0928 16:40:42.484228 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:42.484360 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":274,"threads_started":1}
I20260928 16:40:42.501507 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 26308
I20260928 16:40:42.523713 26785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:42.502935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40352 (local address 127.10.247.193:37211)
0928 16:40:42.503679 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:42.503702 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:42.503761 (+ 59us) server_negotiation.cc:407] Connection header received
0928 16:40:42.506184 (+ 2423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:42.506199 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:42.506797 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:42.507191 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:42.508932 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:42.510410 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:42.511788 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:42.512559 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:42.512781 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:42.516068 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:42.516078 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:42.516101 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:42.516158 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:42.519461 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:42.520387 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:42.520395 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:42.520404 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:42.520654 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:42.521162 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:42.521168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:42.521171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:42.521896 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:42.522249 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:42.522772 (+ 523us) server_negotiation.cc:299] Negotiation successful
0928 16:40:42.523325 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":363,"threads_started":1}
I20260928 16:40:42.525281 26786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:42.524347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45029 (local address 127.10.247.195:59711)
0928 16:40:42.524897 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:42.525027 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":291,"threads_started":1}
I20260928 16:40:42.525719 26467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:42.620416 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 26308
I20260928 16:40:42.639775 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 26475
I20260928 16:40:42.661576 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:42.641293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57342 (local address 127.10.247.194:46121)
0928 16:40:42.642527 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:42.642544 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:42.642595 (+ 51us) server_negotiation.cc:407] Connection header received
0928 16:40:42.644495 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:42.644510 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:42.645252 (+ 742us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:42.645633 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:42.647401 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:42.649010 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:42.650438 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:42.651294 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:42.651455 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:42.654618 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:42.654627 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:42.654650 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:42.654715 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:42.657927 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:42.658706 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:42.658715 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:42.658724 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:42.658931 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:42.659284 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:42.659288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:42.659290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:42.660022 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:42.660302 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:42.660777 (+ 475us) server_negotiation.cc:299] Negotiation successful
0928 16:40:42.661227 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1037,"thread_start_us":878,"threads_started":1}
I20260928 16:40:42.663347 26600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:42.753592 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 26475
I20260928 16:40:42.771420 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 26608
I20260928 16:40:42.792681 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:42.772871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57734 (local address 127.10.247.195:43495)
0928 16:40:42.773659 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:42.773676 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:42.773729 (+ 53us) server_negotiation.cc:407] Connection header received
0928 16:40:42.775619 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:42.775632 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:42.776191 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:42.776565 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:42.778154 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:42.779588 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:42.781388 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:42.782192 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:42.782351 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:42.785739 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:42.785748 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:42.785769 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:42.785818 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:42.789167 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:42.789912 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:42.789918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:42.789926 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:42.790145 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:42.790499 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:42.790503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:42.790505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:42.791166 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:42.791433 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:42.791784 (+ 351us) server_negotiation.cc:299] Negotiation successful
0928 16:40:42.792290 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":387,"threads_started":1}
I20260928 16:40:42.794425 26732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:42.885737 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 26608
2026-09-28T16:40:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25517 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:40:42 dist-test-slave-9fn6 krb5kdc[26794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:40:42 dist-test-slave-9fn6 krb5kdc[26794](info): set up 2 sockets
Sep 28 16:40:42 dist-test-slave-9fn6 krb5kdc[26794](info): commencing operation
krb5kdc: starting...
W20260928 16:40:43.930073 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:40:44 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:40:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:40:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:40:44.234117 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45277
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:45277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:44.630481 26807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:44.630905 26807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:44.631042 26807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:44.639302 26807 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 16:40:44.639441 26807 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 16:40:44.639480 26807 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 16:40:44.639551 26807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:40:44.639633 26807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:44.639724 26807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:44.639786 26807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:40:44.639865 26807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:40:44.651374 26807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:45277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:44.653208 26807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:44.654976 26807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:44.668965 26813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:44.669034 26812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:44.669013 26815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:44.669790 26807 server_base.cc:1034] running on GCE node
I20260928 16:40:44.670701 26807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:44.672407 26807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:44.673640 26807 hybrid_clock.cc:648] HybridClock initialized: now 1790613644673544 us; error 59 us; skew 500 ppm
I20260928 16:40:44.674046 26807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:44 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613644, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:44.677763 26807 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:40:44.679569 26807 webserver.cc:460] Webserver started at http://127.10.247.254:35587/ using document root <none> and password file <none>
I20260928 16:40:44.680452 26807 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:44.680554 26807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:44.680938 26807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:44.684701 26807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "725959fc0f6d4e2a8abfe10e14e86bff"
format_stamp: "Formatted at 2026-09-28 16:40:44 on dist-test-slave-9fn6"
server_key: "24156a6136f67ed377dd71fc689c6fd5"
server_key_iv: "e347223c21a9dd98325a01ecb552dafa"
server_key_version: "encryptionkey@0"
I20260928 16:40:44.685658 26807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "725959fc0f6d4e2a8abfe10e14e86bff"
format_stamp: "Formatted at 2026-09-28 16:40:44 on dist-test-slave-9fn6"
server_key: "24156a6136f67ed377dd71fc689c6fd5"
server_key_iv: "e347223c21a9dd98325a01ecb552dafa"
server_key_version: "encryptionkey@0"
I20260928 16:40:44.692256 26807 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:40:44.696705 26822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:44.698603 26807 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:40:44.698823 26807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "725959fc0f6d4e2a8abfe10e14e86bff"
format_stamp: "Formatted at 2026-09-28 16:40:44 on dist-test-slave-9fn6"
server_key: "24156a6136f67ed377dd71fc689c6fd5"
server_key_iv: "e347223c21a9dd98325a01ecb552dafa"
server_key_version: "encryptionkey@0"
I20260928 16:40:44.699086 26807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:44.728931 26807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:44.734213 26807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:44.734530 26807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:44.755323 26807 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:45277
I20260928 16:40:44.755304 26873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:45277 every 8 connection(s)
I20260928 16:40:44.757047 26807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:40:44.761631 26874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:44.766965 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 26807
I20260928 16:40:44.767195 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:40:44.767956 11231 external_mini_cluster.cc:1428] Setting key 0e3f404b1cdc54f95df75bd642b645ff
I20260928 16:40:44.775476 26874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Bootstrap starting.
I20260928 16:40:44.780121 26874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:44.781586 26874 log.cc:824] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:44.785277 26874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: No bootstrap required, opened a new log
Sep 28 16:40:44 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:44.797371 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:44.770382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.10.247.254:45277)
0928 16:40:44.771343 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:44.771361 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:44.771428 (+ 67us) server_negotiation.cc:407] Connection header received
0928 16:40:44.774047 (+ 2619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:44.774087 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:44.774745 (+ 658us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:44.775302 (+ 557us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:44.777147 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:44.779383 (+ 2236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:44.780963 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:44.782105 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:44.782342 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:44.786973 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:44.786987 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:44.787021 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:44.787079 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:44.791589 (+ 4510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:44.792416 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:44.792431 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:44.792443 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:44.792734 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:44.793159 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:44.793164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:44.793168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:44.794132 (+ 964us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:44.794497 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:44.794932 (+ 435us) server_negotiation.cc:299] Negotiation successful
0928 16:40:44.795564 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":379,"threads_started":1}
I20260928 16:40:44.796957 26874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:40:44.797571 26874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:44.797734 26874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725959fc0f6d4e2a8abfe10e14e86bff, State: Initialized, Role: FOLLOWER
I20260928 16:40:44.798437 26874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:40:44.798698 26874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:40:44.798843 26874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:40:44.799037 26874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:44.801780 26874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:40:44.802400 26874 leader_election.cc:302] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 725959fc0f6d4e2a8abfe10e14e86bff; no voters:
I20260928 16:40:44.803062 26874 leader_election.cc:288] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:40:44.803283 26879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:40:44.804086 26879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 1 LEADER]: Becoming Leader. State: Replica: 725959fc0f6d4e2a8abfe10e14e86bff, State: Running, Role: LEADER
I20260928 16:40:44.804913 26879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:40:44.805708 26874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:40:44.808164 26881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 725959fc0f6d4e2a8abfe10e14e86bff. Latest consensus state: current_term: 1 leader_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } } }
I20260928 16:40:44.808873 26880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } } }
I20260928 16:40:44.809383 26881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:44.809389 26880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: This master's current role is: LEADER
I20260928 16:40:44.812927 26889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:40:44.818389 26889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:40:44.829082 26889 catalog_manager.cc:1347] Generated new cluster ID: e50fb48416594b4fb03ca3ef68b484ea
I20260928 16:40:44.829265 26889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:40:44.844959 26889 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:40:44.846684 26889 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:40:44.858246 26889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Generated new TSK 0
I20260928 16:40:44.859390 26889 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:40:44.953431 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45277
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:45.348929 26900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:45.349301 26900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:45.349436 26900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:45.357743 26900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:45.357933 26900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:45.358109 26900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:40:45.369168 26900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:45.370975 26900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:45.372684 26900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:45.387799 26905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:45.387866 26908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:45.387840 26906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:45.388827 26900 server_base.cc:1034] running on GCE node
I20260928 16:40:45.389572 26900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:45.390744 26900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:45.391973 26900 hybrid_clock.cc:648] HybridClock initialized: now 1790613645391899 us; error 61 us; skew 500 ppm
I20260928 16:40:45.392352 26900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:45 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613645, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:45.402091 26900 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:40:45.404076 26900 webserver.cc:460] Webserver started at http://127.10.247.193:44941/ using document root <none> and password file <none>
I20260928 16:40:45.405040 26900 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:45.405159 26900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:45.405483 26900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:45.409202 26900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "93fc5855301b41219e1d385a839506a0"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "3014e251e53142a12de2d8cb9389f8d1"
server_key_iv: "df8e2df5dc94be016ceeb84d3a634fef"
server_key_version: "encryptionkey@0"
I20260928 16:40:45.410044 26900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "93fc5855301b41219e1d385a839506a0"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "3014e251e53142a12de2d8cb9389f8d1"
server_key_iv: "df8e2df5dc94be016ceeb84d3a634fef"
server_key_version: "encryptionkey@0"
I20260928 16:40:45.416090 26900 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:40:45.420151 26915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:45.421795 26900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:40:45.421963 26900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "93fc5855301b41219e1d385a839506a0"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "3014e251e53142a12de2d8cb9389f8d1"
server_key_iv: "df8e2df5dc94be016ceeb84d3a634fef"
server_key_version: "encryptionkey@0"
I20260928 16:40:45.422204 26900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:45.456940 26900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:45.461966 26900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:45.462303 26900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:45.463469 26900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:45.465328 26900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:45.465425 26900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:45.465561 26900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:45.465617 26900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:45.499915 26900 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:44613
I20260928 16:40:45.499951 27028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:44613 every 8 connection(s)
I20260928 16:40:45.501482 26900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:40:45.506518 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 26900
I20260928 16:40:45.506759 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:40:45.507473 11231 external_mini_cluster.cc:1428] Setting key 1a3ec87bcf1b688b07c8f2e1b9a3d2fb
Sep 28 16:40:45 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613645, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:45.523737 27031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:45.504449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:35421 (local address 127.10.247.254:45277)
0928 16:40:45.505055 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:45.505060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:45.506084 (+ 1024us) server_negotiation.cc:407] Connection header received
0928 16:40:45.508347 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:45.508355 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:45.508459 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:45.508590 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:45.510721 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:45.511511 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:45.513165 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:45.513589 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:45.513705 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:45.517484 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:45.517492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:45.517496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:45.517533 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:45.520256 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:45.521069 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:45.521074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:45.521077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:45.521211 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:45.521738 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:45.521742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:45.521744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:45.522076 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:45.522222 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:45.523054 (+ 832us) server_negotiation.cc:299] Negotiation successful
0928 16:40:45.523333 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":267,"threads_started":1}
I20260928 16:40:45.525079 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:45.505112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:35421)
0928 16:40:45.505881 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:45.505934 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:45.508014 (+ 2080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:45.508933 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:45.508946 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:45.509820 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:45.510544 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:45.510564 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:45.511656 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:45.511661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:45.513021 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:45.513035 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:45.513825 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:45.513833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:45.514090 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:45.514784 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:45.514815 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:45.517237 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:45.520394 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:45.520403 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:45.520423 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:45.520936 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:45.521327 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:45.521333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:45.521337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:45.521622 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:45.522205 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:45.522214 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:45.522635 (+ 421us) client_negotiation.cc:769] Sending connection context
0928 16:40:45.522907 (+ 272us) client_negotiation.cc:240] Negotiation successful
0928 16:40:45.523554 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":293,"threads_started":1}
I20260928 16:40:45.527025 27029 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45277
I20260928 16:40:45.527431 27029 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:45.528342 27029 heartbeater.cc:511] Master 127.10.247.254:45277 requested a full tablet report, sending...
I20260928 16:40:45.530933 26839 ts_manager.cc:194] Registered new tserver with Master: 93fc5855301b41219e1d385a839506a0 (127.10.247.193:44613)
I20260928 16:40:45.532711 26839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:35421
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:40:45.570353 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45277
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:40:45.951023 27035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:45.951365 27035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:45.951470 27035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:45.959316 27035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:45.959453 27035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:45.959589 27035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:40:45.970155 27035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:45.971825 27035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:45.973467 27035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:45.988747 27040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:45.988747 27043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:45.988776 27041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:45.989575 27035 server_base.cc:1034] running on GCE node
I20260928 16:40:45.990324 27035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:45.991496 27035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:45.992709 27035 hybrid_clock.cc:648] HybridClock initialized: now 1790613645992647 us; error 50 us; skew 500 ppm
I20260928 16:40:45.993073 27035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:45 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613645, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:45.996901 27035 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:40:45.998745 27035 webserver.cc:460] Webserver started at http://127.10.247.194:42175/ using document root <none> and password file <none>
I20260928 16:40:45.999715 27035 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:45.999866 27035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:46.000248 27035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:46.004416 27035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "d4c100c335d44c38b74794e652ac9863"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "28424b679a627ace179804b8718174f3"
server_key_iv: "53cc8f6548dfe7d8306a9a78b8a234fa"
server_key_version: "encryptionkey@0"
I20260928 16:40:46.005470 27035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "d4c100c335d44c38b74794e652ac9863"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "28424b679a627ace179804b8718174f3"
server_key_iv: "53cc8f6548dfe7d8306a9a78b8a234fa"
server_key_version: "encryptionkey@0"
I20260928 16:40:46.012495 27035 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260928 16:40:46.016923 27050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:46.018651 27035 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260928 16:40:46.018863 27035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "d4c100c335d44c38b74794e652ac9863"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "28424b679a627ace179804b8718174f3"
server_key_iv: "53cc8f6548dfe7d8306a9a78b8a234fa"
server_key_version: "encryptionkey@0"
I20260928 16:40:46.019121 27035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:46.042225 27035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:46.047235 27035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:46.047554 27035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:46.048771 27035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:46.050414 27035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:46.050506 27035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:46.050637 27035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:46.050691 27035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:46.085222 27035 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:34663
I20260928 16:40:46.085273 27162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:34663 every 8 connection(s)
I20260928 16:40:46.086776 27035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:40:46.091334 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 27035
I20260928 16:40:46.091583 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:40:46.092271 11231 external_mini_cluster.cc:1428] Setting key 0268614db04850e43db22e925bab5ed9
Sep 28 16:40:46 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613645, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:46.113212 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.089802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:51769 (local address 127.10.247.254:45277)
0928 16:40:46.090397 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.090404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.091616 (+ 1212us) server_negotiation.cc:407] Connection header received
0928 16:40:46.094065 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.094072 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.094161 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.094289 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:46.096709 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.097618 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.099944 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.100574 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.100770 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.106031 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:46.106039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:46.106043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:46.106098 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:46.109710 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:46.110618 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:46.110625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:46.110629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:46.110785 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:46.111200 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:46.111204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:46.111207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:46.111590 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:46.111731 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.112550 (+ 819us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.112827 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":258,"threads_started":1}
I20260928 16:40:46.114424 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.090452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:51769)
0928 16:40:46.091369 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:46.091442 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:46.093773 (+ 2331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:46.094597 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:46.094612 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:46.095493 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:46.096442 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.096469 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.097793 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.097799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.099738 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.099757 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.100986 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.100995 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.101393 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.102319 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:46.102354 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:46.105706 (+ 3352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:46.109875 (+ 4169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:46.109885 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:46.109903 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:46.110502 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:46.110884 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:46.110888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:46.110890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:46.111099 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:46.111715 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:46.111723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:46.112163 (+ 440us) client_negotiation.cc:769] Sending connection context
0928 16:40:46.112442 (+ 279us) client_negotiation.cc:240] Negotiation successful
0928 16:40:46.112988 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":297,"threads_started":1}
I20260928 16:40:46.116173 27163 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45277
I20260928 16:40:46.116550 27163 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:46.117605 27163 heartbeater.cc:511] Master 127.10.247.254:45277 requested a full tablet report, sending...
I20260928 16:40:46.119516 26839 ts_manager.cc:194] Registered new tserver with Master: d4c100c335d44c38b74794e652ac9863 (127.10.247.194:34663)
I20260928 16:40:46.120473 26839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:51769
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:40:46.165362 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45277
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:40:46.536420 27029 heartbeater.cc:503] Master 127.10.247.254:45277 was elected leader, sending a full tablet report...
W20260928 16:40:46.550988 27169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:40:46.551353 27169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:40:46.551497 27169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:40:46.559422 27169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:40:46.559564 27169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:40:46.559736 27169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:40:46.570487 27169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:45277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:40:46.572216 27169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:40:46.573915 27169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:40:46.589010 27175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:46.589093 27174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:40:46.589113 27177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:40:46.589962 27169 server_base.cc:1034] running on GCE node
I20260928 16:40:46.590723 27169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:40:46.591877 27169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:40:46.593081 27169 hybrid_clock.cc:648] HybridClock initialized: now 1790613646593026 us; error 50 us; skew 500 ppm
I20260928 16:40:46.593444 27169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:40:46 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613646, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:40:46.596987 27169 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:40:46.598752 27169 webserver.cc:460] Webserver started at http://127.10.247.195:39775/ using document root <none> and password file <none>
I20260928 16:40:46.599608 27169 fs_manager.cc:359] Metadata directory not provided
I20260928 16:40:46.599737 27169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:40:46.600101 27169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:40:46.603919 27169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "5b2711cedc194aabb6a432d876c89c9f"
format_stamp: "Formatted at 2026-09-28 16:40:46 on dist-test-slave-9fn6"
server_key: "f25b3d9b9acaa2bb4f14d8597c01b77b"
server_key_iv: "59b201e4f2d90a635875000a2a014a1a"
server_key_version: "encryptionkey@0"
I20260928 16:40:46.605093 27169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "5b2711cedc194aabb6a432d876c89c9f"
format_stamp: "Formatted at 2026-09-28 16:40:46 on dist-test-slave-9fn6"
server_key: "f25b3d9b9acaa2bb4f14d8597c01b77b"
server_key_iv: "59b201e4f2d90a635875000a2a014a1a"
server_key_version: "encryptionkey@0"
I20260928 16:40:46.613535 27169 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260928 16:40:46.618053 27184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:46.619889 27169 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:40:46.620095 27169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "5b2711cedc194aabb6a432d876c89c9f"
format_stamp: "Formatted at 2026-09-28 16:40:46 on dist-test-slave-9fn6"
server_key: "f25b3d9b9acaa2bb4f14d8597c01b77b"
server_key_iv: "59b201e4f2d90a635875000a2a014a1a"
server_key_version: "encryptionkey@0"
I20260928 16:40:46.620364 27169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:40:46.642865 27169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:40:46.648253 27169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:40:46.648591 27169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:40:46.649856 27169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:40:46.651547 27169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:40:46.651633 27169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:46.651782 27169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:40:46.651839 27169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:40:46.685950 27169 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:34147
I20260928 16:40:46.685989 27297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:34147 every 8 connection(s)
I20260928 16:40:46.687460 27169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:40:46.697584 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 27169
I20260928 16:40:46.697778 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:40:46.698328 11231 external_mini_cluster.cc:1428] Setting key d87117b1b0e08891653ef273562b9d51
Sep 28 16:40:46 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613646, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:40:46.709383 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.690396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:44169 (local address 127.10.247.254:45277)
0928 16:40:46.690935 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.690940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.691846 (+ 906us) server_negotiation.cc:407] Connection header received
0928 16:40:46.693898 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.693906 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.694009 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.694119 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:46.695841 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.696821 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.698562 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.699128 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.699265 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.703294 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:46.703300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:46.703304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:46.703343 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:46.706067 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:46.706793 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:46.706799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:46.706802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:46.706938 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:46.707368 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:46.707372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:46.707374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:46.707685 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:46.707844 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.708688 (+ 844us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.708942 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":261,"threads_started":1}
I20260928 16:40:46.710558 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.690873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:44169)
0928 16:40:46.691650 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:46.691695 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:46.693595 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:46.694401 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:46.694409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:46.694992 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:40:46.695656 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.695676 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.696976 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.696979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.698393 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.698407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.699388 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.699395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.699653 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.700391 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:40:46.700422 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:40:46.703073 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:40:46.706204 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:46.706214 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:46.706235 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:46.706678 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:46.707041 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:40:46.707054 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:40:46.707058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:40:46.707260 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:40:46.707811 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:40:46.707821 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:40:46.708249 (+ 428us) client_negotiation.cc:769] Sending connection context
0928 16:40:46.708548 (+ 299us) client_negotiation.cc:240] Negotiation successful
0928 16:40:46.709209 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":293,"threads_started":1}
I20260928 16:40:46.712118 27298 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45277
I20260928 16:40:46.712467 27298 heartbeater.cc:464] Registering TS with master...
I20260928 16:40:46.713465 27298 heartbeater.cc:511] Master 127.10.247.254:45277 requested a full tablet report, sending...
I20260928 16:40:46.715139 26839 ts_manager.cc:194] Registered new tserver with Master: 5b2711cedc194aabb6a432d876c89c9f (127.10.247.195:34147)
I20260928 16:40:46.716073 26839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:44169
I20260928 16:40:46.727013 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:40:46.745579 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.732327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.10.247.254:45277)
0928 16:40:46.732601 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.732630 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.733052 (+ 422us) server_negotiation.cc:407] Connection header received
0928 16:40:46.733236 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.733241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.733339 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.733419 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:46.734956 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.735745 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.737192 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.737640 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.737740 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.740194 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:46.740200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:46.740204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:46.740243 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:46.743041 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:46.743752 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:46.743757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:46.743759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:46.743891 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:46.744176 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:46.744179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:46.744181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:46.744537 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:46.744785 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.745094 (+ 309us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.745295 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 16:40:46.749923 26839 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:40:46.758100 26839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:40:46.795413 27316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.774719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55656 (local address 127.10.247.193:44613)
0928 16:40:46.783230 (+ 8511us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.783241 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.787466 (+ 4225us) server_negotiation.cc:407] Connection header received
0928 16:40:46.787585 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.787593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.787836 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.788031 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:46.789015 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.790454 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.793018 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.794203 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.794393 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.794621 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.794857 (+ 236us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.795043 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8304,"thread_start_us":8126,"threads_started":1}
I20260928 16:40:46.795529 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.773961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:55656)
0928 16:40:46.787197 (+ 13236us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:46.787214 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:46.787332 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:46.788351 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:46.788355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:46.788378 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:46.788817 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.788831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.790587 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.790592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.792819 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.792835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.794393 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.794399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.794597 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.794651 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 16:40:46.794775 (+ 124us) client_negotiation.cc:240] Negotiation successful
0928 16:40:46.794996 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13105,"mutex_wait_us":337,"thread_start_us":7581,"threads_started":1}
I20260928 16:40:46.796232 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.772285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45788 (local address 127.10.247.195:34147)
0928 16:40:46.786436 (+ 14151us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.786444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.786489 (+ 45us) server_negotiation.cc:407] Connection header received
0928 16:40:46.786682 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.786692 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.787021 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.787287 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:46.788158 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.789942 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.793315 (+ 3373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.794747 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.794920 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.795076 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.795662 (+ 586us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.795835 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13964,"thread_start_us":13780,"threads_started":1}
I20260928 16:40:46.796381 27315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.772093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34663 (local address 127.0.0.1:49756)
0928 16:40:46.782159 (+ 10066us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:46.782242 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:46.782516 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:46.787602 (+ 5086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:46.787605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:46.787626 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:46.788138 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.788153 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.789933 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.789938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.792858 (+ 2920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.792873 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.795119 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.795125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.795331 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.795371 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 16:40:46.795464 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 16:40:46.795950 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9654,"mutex_wait_us":180,"thread_start_us":12911,"threads_started":1}
I20260928 16:40:46.797021 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.772516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.10.247.194:34663)
0928 16:40:46.786787 (+ 14271us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.786796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.786822 (+ 26us) server_negotiation.cc:407] Connection header received
0928 16:40:46.786914 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.786920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.787108 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.787287 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:46.788349 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.789799 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.793068 (+ 3269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.794595 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.794768 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.794952 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.795803 (+ 851us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.796004 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14097,"thread_start_us":13967,"threads_started":1}
I20260928 16:40:46.797572 27311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.772123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:34147 (local address 127.0.0.1:45788)
0928 16:40:46.785356 (+ 13233us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:46.785380 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:46.785539 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:46.787313 (+ 1774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:46.787323 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:46.787350 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:46.787888 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.787907 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.790091 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.790096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.793146 (+ 3050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.793161 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.795230 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.795236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.795421 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.795451 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 16:40:46.795527 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 16:40:46.796377 (+ 850us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13074,"mutex_wait_us":1062,"thread_start_us":14801,"threads_started":1}
I20260928 16:40:46.800998 27233 tablet_service.cc:1467] Processing CreateTablet for tablet 6296a252bcbd47769f17b85bc9d45c7a (DEFAULT_TABLE table=test-table [id=44d9701b716a4d238d43fc3fa9064cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:46.801007 26964 tablet_service.cc:1467] Processing CreateTablet for tablet 6296a252bcbd47769f17b85bc9d45c7a (DEFAULT_TABLE table=test-table [id=44d9701b716a4d238d43fc3fa9064cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:46.801234 27098 tablet_service.cc:1467] Processing CreateTablet for tablet 6296a252bcbd47769f17b85bc9d45c7a (DEFAULT_TABLE table=test-table [id=44d9701b716a4d238d43fc3fa9064cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:40:46.802601 27098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6296a252bcbd47769f17b85bc9d45c7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:46.802592 26964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6296a252bcbd47769f17b85bc9d45c7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:46.803435 27233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6296a252bcbd47769f17b85bc9d45c7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:40:46.818589 27319 tablet_bootstrap.cc:492] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: Bootstrap starting.
I20260928 16:40:46.823711 27318 tablet_bootstrap.cc:492] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: Bootstrap starting.
I20260928 16:40:46.823704 27317 tablet_bootstrap.cc:492] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: Bootstrap starting.
I20260928 16:40:46.825954 27319 tablet_bootstrap.cc:654] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:46.827953 27319 log.cc:824] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:46.828068 27317 tablet_bootstrap.cc:654] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:46.829521 27318 tablet_bootstrap.cc:654] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: Neither blocks nor log segments found. Creating new log.
I20260928 16:40:46.829531 27317 log.cc:824] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:46.830915 27318 log.cc:824] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: Log is configured to *not* fsync() on all Append() calls
I20260928 16:40:46.832335 27319 tablet_bootstrap.cc:492] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: No bootstrap required, opened a new log
I20260928 16:40:46.832743 27317 tablet_bootstrap.cc:492] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: No bootstrap required, opened a new log
I20260928 16:40:46.832757 27319 ts_tablet_manager.cc:1397] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260928 16:40:46.833011 27317 ts_tablet_manager.cc:1397] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260928 16:40:46.834287 27318 tablet_bootstrap.cc:492] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: No bootstrap required, opened a new log
I20260928 16:40:46.834612 27318 ts_tablet_manager.cc:1397] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260928 16:40:46.844152 27317 raft_consensus.cc:348] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:40:46.844180 27319 raft_consensus.cc:348] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:40:46.844689 27319 raft_consensus.cc:374] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:46.844709 27317 raft_consensus.cc:374] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:46.844808 27317 raft_consensus.cc:729] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2711cedc194aabb6a432d876c89c9f, State: Initialized, Role: FOLLOWER
I20260928 16:40:46.844834 27319 raft_consensus.cc:729] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fc5855301b41219e1d385a839506a0, State: Initialized, Role: FOLLOWER
I20260928 16:40:46.845331 27318 raft_consensus.cc:348] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:40:46.845569 27317 consensus_queue.cc:260] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:40:46.845572 27319 consensus_queue.cc:260] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:40:46.845798 27318 raft_consensus.cc:374] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:40:46.845920 27318 raft_consensus.cc:729] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c100c335d44c38b74794e652ac9863, State: Initialized, Role: FOLLOWER
I20260928 16:40:46.846549 27318 consensus_queue.cc:260] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:40:46.847946 27163 heartbeater.cc:503] Master 127.10.247.254:45277 was elected leader, sending a full tablet report...
I20260928 16:40:46.848903 27318 ts_tablet_manager.cc:1428] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260928 16:40:46.850703 27319 ts_tablet_manager.cc:1428] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260928 16:40:46.852662 27298 heartbeater.cc:503] Master 127.10.247.254:45277 was elected leader, sending a full tablet report...
I20260928 16:40:46.852864 27317 ts_tablet_manager.cc:1428] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: Time spent starting tablet: real 0.020s user 0.009s sys 0.008s
I20260928 16:40:46.855867 27325 raft_consensus.cc:482] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:40:46.856258 27325 raft_consensus.cc:504] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:40:46.858249 27325 leader_election.cc:288] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2711cedc194aabb6a432d876c89c9f (127.10.247.195:34147), 93fc5855301b41219e1d385a839506a0 (127.10.247.193:44613)
I20260928 16:40:46.865559 27316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.858705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:59291 (local address 127.10.247.193:44613)
0928 16:40:46.858897 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.858902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.860287 (+ 1385us) server_negotiation.cc:407] Connection header received
0928 16:40:46.860431 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.860437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.860518 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.860601 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:46.861203 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.862037 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.863778 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.864562 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.864758 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.864844 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.865158 (+ 314us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.865301 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:40:46.865561 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.858712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:49495 (local address 127.10.247.195:34147)
0928 16:40:46.859024 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.859028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.859571 (+ 543us) server_negotiation.cc:407] Connection header received
0928 16:40:46.859777 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.859783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.859869 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.859963 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:40:46.860673 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.861665 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.863695 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.864554 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.864757 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.864840 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.865174 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.865306 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260928 16:40:46.865747 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.858493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.10.247.194:59291)
0928 16:40:46.860149 (+ 1656us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:46.860192 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:46.860306 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:46.860657 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:46.860661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:46.860686 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:46.861068 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.861080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.862186 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.862190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.863630 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.863646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.864823 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.864829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.864990 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.865030 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 16:40:46.865105 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 16:40:46.865231 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1524,"mutex_wait_us":30,"thread_start_us":1329,"threads_started":1}
I20260928 16:40:46.865775 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.858487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:34147 (local address 127.10.247.194:49495)
0928 16:40:46.859461 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0928 16:40:46.859477 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:40:46.859637 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:40:46.860012 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:40:46.860018 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:40:46.860050 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:40:46.860465 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.860478 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.861799 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.861805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.863538 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:40:46.863551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.864854 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.864861 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:40:46.865027 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.865051 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 16:40:46.865136 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 16:40:46.865287 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":36,"thread_start_us":668,"threads_started":1}
I20260928 16:40:46.866895 27253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6296a252bcbd47769f17b85bc9d45c7a" candidate_uuid: "d4c100c335d44c38b74794e652ac9863" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2711cedc194aabb6a432d876c89c9f" is_pre_election: true
I20260928 16:40:46.867465 27253 raft_consensus.cc:2454] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c100c335d44c38b74794e652ac9863 in term 0.
I20260928 16:40:46.867689 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6296a252bcbd47769f17b85bc9d45c7a" candidate_uuid: "d4c100c335d44c38b74794e652ac9863" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fc5855301b41219e1d385a839506a0" is_pre_election: true
I20260928 16:40:46.868186 27053 leader_election.cc:302] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2711cedc194aabb6a432d876c89c9f, d4c100c335d44c38b74794e652ac9863; no voters:
I20260928 16:40:46.868283 26984 raft_consensus.cc:2454] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c100c335d44c38b74794e652ac9863 in term 0.
I20260928 16:40:46.868640 27325 raft_consensus.cc:2781] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:40:46.868814 27325 raft_consensus.cc:482] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:40:46.868916 27325 raft_consensus.cc:3037] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:46.871608 27325 raft_consensus.cc:504] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:40:46.872222 27325 leader_election.cc:288] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [CANDIDATE]: Term 1 election: Requested vote from peers 5b2711cedc194aabb6a432d876c89c9f (127.10.247.195:34147), 93fc5855301b41219e1d385a839506a0 (127.10.247.193:44613)
I20260928 16:40:46.872737 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6296a252bcbd47769f17b85bc9d45c7a" candidate_uuid: "d4c100c335d44c38b74794e652ac9863" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fc5855301b41219e1d385a839506a0"
I20260928 16:40:46.872738 27253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6296a252bcbd47769f17b85bc9d45c7a" candidate_uuid: "d4c100c335d44c38b74794e652ac9863" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2711cedc194aabb6a432d876c89c9f"
I20260928 16:40:46.872931 26984 raft_consensus.cc:3037] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:46.872936 27253 raft_consensus.cc:3037] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:40:46.875283 26984 raft_consensus.cc:2454] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c100c335d44c38b74794e652ac9863 in term 1.
I20260928 16:40:46.875299 27253 raft_consensus.cc:2454] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c100c335d44c38b74794e652ac9863 in term 1.
I20260928 16:40:46.875691 27053 leader_election.cc:302] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2711cedc194aabb6a432d876c89c9f, d4c100c335d44c38b74794e652ac9863; no voters:
I20260928 16:40:46.876020 27325 raft_consensus.cc:2781] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:40:46.876487 27325 raft_consensus.cc:686] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 1 LEADER]: Becoming Leader. State: Replica: d4c100c335d44c38b74794e652ac9863, State: Running, Role: LEADER
I20260928 16:40:46.877053 27325 consensus_queue.cc:237] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:40:46.882253 26838 catalog_manager.cc:5495] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4c100c335d44c38b74794e652ac9863 (127.10.247.194). New cstate: current_term: 1 leader_uuid: "d4c100c335d44c38b74794e652ac9863" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } health_report { overall_health: UNKNOWN } } }
I20260928 16:40:46.903148 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.897298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49760 (local address 127.10.247.194:34663)
0928 16:40:46.897496 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.897501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.897521 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:40:46.897733 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.897737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.897846 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.897989 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:40:46.898358 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.899190 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.900693 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.901136 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.901241 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.901843 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:40:46.902564 (+ 721us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:40:46.902683 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.902776 (+ 93us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.902864 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:40:46.913542 27253 raft_consensus.cc:1261] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [term 1 FOLLOWER]: Refusing update from remote peer d4c100c335d44c38b74794e652ac9863: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:46.913542 26984 raft_consensus.cc:1261] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [term 1 FOLLOWER]: Refusing update from remote peer d4c100c335d44c38b74794e652ac9863: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:40:46.914657 27328 consensus_queue.cc:1035] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:40:46.915098 27325 consensus_queue.cc:1035] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:40:46.929509 27331 mvcc.cc:205] Tried to move back new op lower bound from 7334353497745743872 to 7334353497612673024. Current Snapshot: MvccSnapshot[applied={T|T < 7334353497745743872}]
I20260928 16:40:46.933539 27332 mvcc.cc:205] Tried to move back new op lower bound from 7334353497745743872 to 7334353497612673024. Current Snapshot: MvccSnapshot[applied={T|T < 7334353497745743872}]
I20260928 16:40:46.946307 26838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:40:46.946702 26838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:40:46.951186 26838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Sending DeleteTablet for 3 replicas of tablet 6296a252bcbd47769f17b85bc9d45c7a
I20260928 16:40:46.952595 27098 tablet_service.cc:1514] Processing DeleteTablet for tablet 6296a252bcbd47769f17b85bc9d45c7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:46 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:49756
I20260928 16:40:46.952589 27233 tablet_service.cc:1514] Processing DeleteTablet for tablet 6296a252bcbd47769f17b85bc9d45c7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:46 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:45788
I20260928 16:40:46.952651 26964 tablet_service.cc:1514] Processing DeleteTablet for tablet 6296a252bcbd47769f17b85bc9d45c7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:40:46 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:55656
I20260928 16:40:46.953956 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 26900
I20260928 16:40:46.969214 27342 tablet_replica.cc:330] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: stopping tablet replica
I20260928 16:40:46.970220 27342 raft_consensus.cc:2229] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:40:46.971412 27342 raft_consensus.cc:2258] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:46.973059 27341 tablet_replica.cc:330] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: stopping tablet replica
I20260928 16:40:46.973663 27341 raft_consensus.cc:2229] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:40:46.974334 27341 raft_consensus.cc:2258] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:46.974771 27343 tablet_replica.cc:330] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: stopping tablet replica
I20260928 16:40:46.975306 27342 ts_tablet_manager.cc:1905] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:46.975291 27343 raft_consensus.cc:2229] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:40:46.975821 27343 raft_consensus.cc:2258] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:40:46.978282 27343 ts_tablet_manager.cc:1905] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:40:46.978281 27341 ts_tablet_manager.cc:1905] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 16:40:46 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:40:46.983861 27342 ts_tablet_manager.cc:1918] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:46.984236 27342 log.cc:1196] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/6296a252bcbd47769f17b85bc9d45c7a
I20260928 16:40:46.985383 27342 ts_tablet_manager.cc:1939] T 6296a252bcbd47769f17b85bc9d45c7a P d4c100c335d44c38b74794e652ac9863: Deleting consensus metadata
I20260928 16:40:46.986876 27341 ts_tablet_manager.cc:1918] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:46.987129 27341 log.cc:1196] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/6296a252bcbd47769f17b85bc9d45c7a
I20260928 16:40:46.987720 27341 ts_tablet_manager.cc:1939] T 6296a252bcbd47769f17b85bc9d45c7a P 93fc5855301b41219e1d385a839506a0: Deleting consensus metadata
I20260928 16:40:46.988224 26823 catalog_manager.cc:4841] TS d4c100c335d44c38b74794e652ac9863 (127.10.247.194:34663): tablet 6296a252bcbd47769f17b85bc9d45c7a (table test-table [id=44d9701b716a4d238d43fc3fa9064cf1]) successfully deleted
I20260928 16:40:46.989905 26826 catalog_manager.cc:4841] TS 93fc5855301b41219e1d385a839506a0 (127.10.247.193:44613): tablet 6296a252bcbd47769f17b85bc9d45c7a (table test-table [id=44d9701b716a4d238d43fc3fa9064cf1]) successfully deleted
I20260928 16:40:46.990281 27343 ts_tablet_manager.cc:1918] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:40:46.990453 27343 log.cc:1196] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/6296a252bcbd47769f17b85bc9d45c7a
I20260928 16:40:46.990902 27343 ts_tablet_manager.cc:1939] T 6296a252bcbd47769f17b85bc9d45c7a P 5b2711cedc194aabb6a432d876c89c9f: Deleting consensus metadata
I20260928 16:40:46.992785 26825 catalog_manager.cc:4841] TS 5b2711cedc194aabb6a432d876c89c9f (127.10.247.195:34147): tablet 6296a252bcbd47769f17b85bc9d45c7a (table test-table [id=44d9701b716a4d238d43fc3fa9064cf1]) successfully deleted
I20260928 16:40:46.998754 27316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:46.955633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55668 (local address 127.10.247.193:44613)
0928 16:40:46.955869 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:46.955877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:46.961677 (+ 5800us) server_negotiation.cc:407] Connection header received
0928 16:40:46.963039 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:46.963048 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:46.963174 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:46.963263 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:46.968008 (+ 4745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.969300 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.972468 (+ 3168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:46.973269 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:46.973428 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:46.990339 (+ 16911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:46.990358 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:46.990381 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:46.990440 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:46.994982 (+ 4542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:46.996029 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:46.996042 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:46.996046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:46.996269 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:46.996796 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:46.996802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:46.996806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:46.997387 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:46.997749 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:46.998018 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 16:40:46.998304 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:40:46.999577 27024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:47.102147 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 26900
I20260928 16:40:47.129844 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 27035
Sep 28 16:40:47 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:40:47.149883 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:47.130830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.10.247.194:34663)
0928 16:40:47.131049 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:47.131054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:47.131120 (+ 66us) server_negotiation.cc:407] Connection header received
0928 16:40:47.131347 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:47.131351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:47.131430 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:47.131519 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:47.133608 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:47.134540 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:47.136549 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:47.137155 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:47.137285 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:47.143043 (+ 5758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:47.143063 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:47.143087 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:47.143143 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:47.146676 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:47.147619 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:47.147628 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:47.147631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:47.147802 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:47.148210 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:47.148213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:47.148215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:47.148705 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:47.149009 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:47.149344 (+ 335us) server_negotiation.cc:299] Negotiation successful
0928 16:40:47.149556 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260928 16:40:47.150718 27158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:47.292860 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 27035
I20260928 16:40:47.312005 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 27169
Sep 28 16:40:47 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:40:47.329025 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:40:47.312946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45790 (local address 127.10.247.195:34147)
0928 16:40:47.313134 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 16:40:47.313139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:40:47.313218 (+ 79us) server_negotiation.cc:407] Connection header received
0928 16:40:47.313394 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:40:47.313399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:40:47.313488 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:40:47.313579 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:40:47.315121 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:47.316277 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:47.317761 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:40:47.318202 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:40:47.318302 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:40:47.323156 (+ 4854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:40:47.323178 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:40:47.323199 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:40:47.323241 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:40:47.326328 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:47.327091 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:47.327099 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:47.327101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:47.327244 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:40:47.327555 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:40:47.327558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:40:47.327560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:40:47.327960 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:40:47.328210 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0928 16:40:47.328451 (+ 241us) server_negotiation.cc:299] Negotiation successful
0928 16:40:47.328725 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 16:40:47.329820 27293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:47.426828 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 27169
I20260928 16:40:47.443485 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 26807
I20260928 16:40:47.444988 26869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:40:47.570426 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 26807
I20260928 16:41:07.593027 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45277
--webserver_interface=127.10.247.254
--webserver_port=35587
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:45277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:41:07.979952 27349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:07.980423 27349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:07.980598 27349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:07.989216 27349 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 16:41:07.989387 27349 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 16:41:07.989449 27349 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 16:41:07.989527 27349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:41:07.989614 27349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:07.989734 27349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:07.989800 27349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:41:07.989882 27349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:41:08.001725 27349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:45277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=35587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:08.003686 27349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:08.005604 27349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:08.034497 27356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:08.034582 27355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:08.034531 27358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:08.035575 27349 server_base.cc:1034] running on GCE node
I20260928 16:41:08.036690 27349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:08.038779 27349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:08.040138 27349 hybrid_clock.cc:648] HybridClock initialized: now 1790613668040008 us; error 67 us; skew 500 ppm
I20260928 16:41:08.040691 27349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:08 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613668, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:08.049165 27349 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:41:08.051530 27349 webserver.cc:460] Webserver started at http://127.10.247.254:35587/ using document root <none> and password file <none>
I20260928 16:41:08.052733 27349 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:08.052904 27349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:08.061040 27349 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 16:41:08.065177 27365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:08.067390 27349 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260928 16:41:08.067662 27349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "725959fc0f6d4e2a8abfe10e14e86bff"
format_stamp: "Formatted at 2026-09-28 16:40:44 on dist-test-slave-9fn6"
server_key: "24156a6136f67ed377dd71fc689c6fd5"
server_key_iv: "e347223c21a9dd98325a01ecb552dafa"
server_key_version: "encryptionkey@0"
I20260928 16:41:08.069147 27349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:08.100842 27349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:08.106495 27349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:08.106845 27349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:08.129312 27349 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:45277
I20260928 16:41:08.129328 27417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:45277 every 8 connection(s)
I20260928 16:41:08.131471 27349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:41:08.135185 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 27349
I20260928 16:41:08.136174 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:44613
--local_ip_for_outbound_sockets=127.10.247.193
--tserver_master_addrs=127.10.247.254:45277
--webserver_port=44941
--webserver_interface=127.10.247.193
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:41:08.141199 27418 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:41:08.145566 27418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Bootstrap starting.
I20260928 16:41:08.170549 27418 log.cc:824] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:08.187376 27418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:41:08.188097 27418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Bootstrap complete.
I20260928 16:41:08.199008 27418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:41:08.200208 27418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725959fc0f6d4e2a8abfe10e14e86bff, State: Initialized, Role: FOLLOWER
I20260928 16:41:08.201128 27418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:41:08.201418 27418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:41:08.201560 27418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:41:08.201756 27418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 1 FOLLOWER]: Advancing to term 2
I20260928 16:41:08.206545 27418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:41:08.207219 27418 leader_election.cc:302] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 725959fc0f6d4e2a8abfe10e14e86bff; no voters:
I20260928 16:41:08.207985 27418 leader_election.cc:288] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 16:41:08.208166 27422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 2 FOLLOWER]: Leader election won for term 2
I20260928 16:41:08.209410 27422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 2 LEADER]: Becoming Leader. State: Replica: 725959fc0f6d4e2a8abfe10e14e86bff, State: Running, Role: LEADER
I20260928 16:41:08.210240 27422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:41:08.210722 27418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:41:08.213125 27424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 725959fc0f6d4e2a8abfe10e14e86bff. Latest consensus state: current_term: 2 leader_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } } }
I20260928 16:41:08.213565 27424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:08.214243 27423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } } }
I20260928 16:41:08.214571 27423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:08.225672 27427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:41:08.236918 27427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:41:08.238678 27427 catalog_manager.cc:1259] Loaded cluster ID: e50fb48416594b4fb03ca3ef68b484ea
I20260928 16:41:08.238885 27427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:41:08.243561 27427 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:41:08.245566 27427 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Loaded TSK: 0
I20260928 16:41:08.286896 27427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Generated new TSK 1
I20260928 16:41:08.287457 27427 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 16:41:08.562438 27420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:08.562860 27420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:08.563093 27420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:08.571530 27420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:08.571712 27420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:08.571889 27420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:41:08.583681 27420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:44613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=44941
--tserver_master_addrs=127.10.247.254:45277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:08.585620 27420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:08.587378 27420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:08.603705 27446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:08.604112 27445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:08.604172 27448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:08.605067 27420 server_base.cc:1034] running on GCE node
I20260928 16:41:08.605855 27420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:08.607092 27420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:08.608333 27420 hybrid_clock.cc:648] HybridClock initialized: now 1790613668608244 us; error 88 us; skew 500 ppm
I20260928 16:41:08.608769 27420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:08 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613668, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:08.612802 27420 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:41:08.614842 27420 webserver.cc:460] Webserver started at http://127.10.247.193:44941/ using document root <none> and password file <none>
I20260928 16:41:08.615760 27420 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:08.615916 27420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:08.623100 27420 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:41:08.626888 27455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:08.628890 27420 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260928 16:41:08.629129 27420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "93fc5855301b41219e1d385a839506a0"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "3014e251e53142a12de2d8cb9389f8d1"
server_key_iv: "df8e2df5dc94be016ceeb84d3a634fef"
server_key_version: "encryptionkey@0"
I20260928 16:41:08.630544 27420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:08.660652 27420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:08.665773 27420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:08.666087 27420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:08.667301 27420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:08.668962 27420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:08.669066 27420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:08.669181 27420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:08.669255 27420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:08.704437 27420 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:44613
I20260928 16:41:08.704489 27567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:44613 every 8 connection(s)
I20260928 16:41:08.706279 27420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:41:08.711623 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 27420
I20260928 16:41:08.712637 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:34663
--local_ip_for_outbound_sockets=127.10.247.194
--tserver_master_addrs=127.10.247.254:45277
--webserver_port=42175
--webserver_interface=127.10.247.194
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:41:08 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613668, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:41:08.737923 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:08.710039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:40819)
0928 16:41:08.711691 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:08.711758 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:08.714480 (+ 2722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:08.716379 (+ 1899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:08.716391 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:08.717293 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:08.718165 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:08.718196 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:08.720090 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:08.720095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:08.721996 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:08.722014 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:08.723211 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:08.723218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:08.723529 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:08.724512 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:41:08.724547 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:08.727485 (+ 2938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:08.732233 (+ 4748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:08.732243 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:08.732260 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:08.733033 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:08.733732 (+ 699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:08.733736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:08.733739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:08.733978 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:08.735100 (+ 1122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:08.735108 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:08.735528 (+ 420us) client_negotiation.cc:769] Sending connection context
0928 16:41:08.735841 (+ 313us) client_negotiation.cc:240] Negotiation successful
0928 16:41:08.736523 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":360,"threads_started":1}
I20260928 16:41:08.737941 27570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:08.709760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:40819 (local address 127.10.247.254:45277)
0928 16:41:08.710483 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:08.710497 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:08.711995 (+ 1498us) server_negotiation.cc:407] Connection header received
0928 16:41:08.715318 (+ 3323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:08.715334 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:08.715796 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:08.716173 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:08.718419 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:08.719897 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:08.722178 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:08.722918 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:08.723098 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:08.727889 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:08.727903 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:08.727930 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:08.727989 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:08.731957 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:08.733225 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:08.733237 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:08.733248 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:08.733553 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:08.734116 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:08.734120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:08.734123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:08.734883 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:08.735107 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:08.736115 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0928 16:41:08.736602 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":332,"threads_started":1}
I20260928 16:41:08.741564 27568 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45277
I20260928 16:41:08.742013 27568 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:08.743196 27568 heartbeater.cc:511] Master 127.10.247.254:45277 requested a full tablet report, sending...
I20260928 16:41:08.746284 27382 ts_manager.cc:194] Registered new tserver with Master: 93fc5855301b41219e1d385a839506a0 (127.10.247.193:44613)
I20260928 16:41:08.747972 27382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:40819
W20260928 16:41:09.110226 27572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:09.110607 27572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:09.110762 27572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:09.118837 27572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:09.119004 27572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:09.119182 27572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:41:09.130707 27572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:34663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=42175
--tserver_master_addrs=127.10.247.254:45277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:09.132331 27572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:09.134088 27572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:09.149693 27577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:09.149852 27578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:09.149909 27580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:09.150919 27572 server_base.cc:1034] running on GCE node
I20260928 16:41:09.151683 27572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:09.152921 27572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:09.154161 27572 hybrid_clock.cc:648] HybridClock initialized: now 1790613669154103 us; error 51 us; skew 500 ppm
I20260928 16:41:09.154568 27572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:09 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613669, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:09.158538 27572 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:41:09.160359 27572 webserver.cc:460] Webserver started at http://127.10.247.194:42175/ using document root <none> and password file <none>
I20260928 16:41:09.161355 27572 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:09.161505 27572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:09.168305 27572 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260928 16:41:09.171782 27587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:09.173624 27572 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260928 16:41:09.173834 27572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "d4c100c335d44c38b74794e652ac9863"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "28424b679a627ace179804b8718174f3"
server_key_iv: "53cc8f6548dfe7d8306a9a78b8a234fa"
server_key_version: "encryptionkey@0"
I20260928 16:41:09.175156 27572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:09.195515 27572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:09.200415 27572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:09.200730 27572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:09.201947 27572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:09.203661 27572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:09.203761 27572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:09.203938 27572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:09.204074 27572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:09.240523 27572 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:34663
I20260928 16:41:09.240574 27699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:34663 every 8 connection(s)
I20260928 16:41:09.242524 27572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:41:09.243556 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 27572
I20260928 16:41:09.244640 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:34147
--local_ip_for_outbound_sockets=127.10.247.195
--tserver_master_addrs=127.10.247.254:45277
--webserver_port=39775
--webserver_interface=127.10.247.195
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:41:09 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613669, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:41:09.268034 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.246281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:58727 (local address 127.10.247.254:45277)
0928 16:41:09.246853 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.246858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.248284 (+ 1426us) server_negotiation.cc:407] Connection header received
0928 16:41:09.250529 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.250536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.250647 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.250761 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:09.252923 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.253767 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.255520 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.255970 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.256099 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.260933 (+ 4834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:09.260941 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:09.260946 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:09.261000 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:09.263951 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:09.265148 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:09.265153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:09.265156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:09.265318 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:09.265835 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:09.265838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:09.265840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:09.266207 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:09.266340 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:09.267422 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0928 16:41:09.267704 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":270,"threads_started":1}
I20260928 16:41:09.269732 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.247150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:58727)
0928 16:41:09.248077 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:09.248132 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:09.250166 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:09.251102 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:09.251114 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:09.251928 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:09.252722 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.252744 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.253947 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.253952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.255360 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.255373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.256235 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.256244 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.256629 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.257486 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:41:09.257518 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:09.260529 (+ 3011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:09.264155 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:09.264167 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:09.264193 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:09.264973 (+ 780us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:09.265426 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:09.265430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:09.265433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:09.265733 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:09.266336 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:09.266346 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:09.266923 (+ 577us) client_negotiation.cc:769] Sending connection context
0928 16:41:09.267267 (+ 344us) client_negotiation.cc:240] Negotiation successful
0928 16:41:09.267961 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":364,"threads_started":1}
I20260928 16:41:09.271799 27700 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45277
I20260928 16:41:09.272210 27700 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:09.273222 27700 heartbeater.cc:511] Master 127.10.247.254:45277 requested a full tablet report, sending...
I20260928 16:41:09.275257 27382 ts_manager.cc:194] Registered new tserver with Master: d4c100c335d44c38b74794e652ac9863 (127.10.247.194:34663)
I20260928 16:41:09.276233 27382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:58727
W20260928 16:41:09.641085 27704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:09.641499 27704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:09.641649 27704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:09.650097 27704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:09.650269 27704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:09.650439 27704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:41:09.661608 27704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:34147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=39775
--tserver_master_addrs=127.10.247.254:45277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:09.663427 27704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:09.665112 27704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:09.680558 27709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:09.680711 27712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:09.680589 27710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:09.681696 27704 server_base.cc:1034] running on GCE node
I20260928 16:41:09.682437 27704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:09.683604 27704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:09.684861 27704 hybrid_clock.cc:648] HybridClock initialized: now 1790613669684788 us; error 54 us; skew 500 ppm
I20260928 16:41:09.685256 27704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:09 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613669, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:09.688933 27704 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:41:09.690923 27704 webserver.cc:460] Webserver started at http://127.10.247.195:39775/ using document root <none> and password file <none>
I20260928 16:41:09.691857 27704 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:09.692013 27704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:09.698665 27704 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:41:09.702077 27719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:09.703966 27704 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260928 16:41:09.704205 27704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "5b2711cedc194aabb6a432d876c89c9f"
format_stamp: "Formatted at 2026-09-28 16:40:46 on dist-test-slave-9fn6"
server_key: "f25b3d9b9acaa2bb4f14d8597c01b77b"
server_key_iv: "59b201e4f2d90a635875000a2a014a1a"
server_key_version: "encryptionkey@0"
I20260928 16:41:09.705840 27704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:09.731449 27704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:09.737174 27704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:09.737497 27704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:09.738754 27704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:09.740424 27704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:09.740523 27704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:09.740672 27704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:09.740746 27704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:09.752519 27568 heartbeater.cc:503] Master 127.10.247.254:45277 was elected leader, sending a full tablet report...
I20260928 16:41:09.773483 27704 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:34147
I20260928 16:41:09.773526 27831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:34147 every 8 connection(s)
I20260928 16:41:09.775244 27704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:41:09.782961 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 27704
Sep 28 16:41:09 dist-test-slave-9fn6 krb5kdc[26794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613669, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:41:09.801373 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.778230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:48411 (local address 127.10.247.254:45277)
0928 16:41:09.778799 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.778806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.779954 (+ 1148us) server_negotiation.cc:407] Connection header received
0928 16:41:09.782145 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.782150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.782256 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.782394 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:09.784280 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.785451 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.787881 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.788635 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.788780 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.793269 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:09.793277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:09.793283 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:09.793330 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:09.796824 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:09.797956 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:09.797960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:09.797963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:09.798108 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:09.798744 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:09.798749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:09.798752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:09.799175 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:09.799338 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:09.800565 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0928 16:41:09.800893 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":394,"spinlock_wait_cycles":4864,"thread_start_us":233,"threads_started":1}
I20260928 16:41:09.802212 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.784128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.10.247.254:45277)
0928 16:41:09.784787 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.784793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.785131 (+ 338us) server_negotiation.cc:407] Connection header received
0928 16:41:09.785296 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.785301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.785387 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.785515 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:09.787681 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.788887 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.790797 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.791288 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.791392 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.794853 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:09.794863 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:09.794867 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:09.794926 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:09.798409 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:09.799432 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:09.799437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:09.799440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:09.799598 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:09.800101 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:09.800106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:09.800109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:09.800555 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:09.800880 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:09.801226 (+ 346us) server_negotiation.cc:299] Negotiation successful
0928 16:41:09.801616 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":500,"spinlock_wait_cycles":11008,"thread_start_us":312,"threads_started":1}
I20260928 16:41:09.803136 27835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.778897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:48411)
0928 16:41:09.779741 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:09.779790 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:09.781813 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:09.782642 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:09.782652 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:09.783334 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:09.784094 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.784116 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.785621 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.785627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.787750 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.787765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.788885 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.788891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.789219 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.790031 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:41:09.790100 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:09.792999 (+ 2899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:09.796996 (+ 3997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:09.797011 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:09.797040 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:09.797813 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:09.798251 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:09.798258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:09.798262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:09.798624 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:09.799361 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:09.799373 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:09.800023 (+ 650us) client_negotiation.cc:769] Sending connection context
0928 16:41:09.800476 (+ 453us) client_negotiation.cc:240] Negotiation successful
0928 16:41:09.801249 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":287,"threads_started":1}
I20260928 16:41:09.804819 27832 heartbeater.cc:347] Connected to a master server at 127.10.247.254:45277
I20260928 16:41:09.805230 27832 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:09.806205 27832 heartbeater.cc:511] Master 127.10.247.254:45277 requested a full tablet report, sending...
I20260928 16:41:09.808074 27382 ts_manager.cc:194] Registered new tserver with Master: 5b2711cedc194aabb6a432d876c89c9f (127.10.247.195:34147)
I20260928 16:41:09.809131 27382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:48411
I20260928 16:41:09.818740 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:41:09.826869 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.819976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.10.247.254:45277)
0928 16:41:09.820236 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.820241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.820737 (+ 496us) server_negotiation.cc:407] Connection header received
0928 16:41:09.820975 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.820978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.821191 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.821341 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:41:09.821880 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.822703 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.824425 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.824982 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.825110 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.825771 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:41:09.826119 (+ 348us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0928 16:41:09.826583 (+ 464us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49574: token expired
Metrics: {"server-negotiator.queue_time_us":102}
W20260928 16:41:09.827025 27837 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49574: token expired
W20260928 16:41:09.827770 27304 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.247.254:45277)
I20260928 16:41:09.844395 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.828316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.10.247.254:45277)
0928 16:41:09.828531 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.828537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.828649 (+ 112us) server_negotiation.cc:407] Connection header received
0928 16:41:09.828852 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.828858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.828985 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.829086 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:09.830916 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.831965 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.833684 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.834181 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.834314 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.837737 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:09.837743 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:09.837747 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:09.837795 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:09.840855 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:09.841794 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:09.841799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:09.841802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:09.841996 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:09.842463 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:09.842468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:09.842470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:09.842924 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:09.843225 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:09.843673 (+ 448us) server_negotiation.cc:299] Negotiation successful
0928 16:41:09.843982 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 16:41:09.849215 27382 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:41:09.858044 27382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:41:09.895694 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.874001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46828 (local address 127.10.247.194:34663)
0928 16:41:09.887997 (+ 13996us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.888006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.888035 (+ 29us) server_negotiation.cc:407] Connection header received
0928 16:41:09.888147 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.888156 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.888397 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.888667 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:09.889230 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.890715 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.893131 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.894184 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.894358 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.894589 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:09.895132 (+ 543us) server_negotiation.cc:299] Negotiation successful
0928 16:41:09.895333 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13839,"thread_start_us":13692,"threads_started":1}
I20260928 16:41:09.895780 27848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.877876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:42532)
0928 16:41:09.886862 (+ 8986us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:09.886879 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:09.887034 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:09.888048 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:09.888066 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:09.888100 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:09.888455 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.888469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.890329 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.890334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.892435 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.892453 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.894859 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.894864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.895029 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.895055 (+ 26us) client_negotiation.cc:769] Sending connection context
0928 16:41:09.895129 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 16:41:09.895237 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8800,"mutex_wait_us":102,"thread_start_us":8609,"threads_started":1}
I20260928 16:41:09.895787 27844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.873591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:34147 (local address 127.0.0.1:36184)
0928 16:41:09.885329 (+ 11738us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:09.885350 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:09.885498 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:09.886618 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:09.886634 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:09.886682 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:09.887338 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.887356 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.889565 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.889571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.892460 (+ 2889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.892477 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.894519 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.894526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.894709 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.894771 (+ 62us) client_negotiation.cc:769] Sending connection context
0928 16:41:09.894915 (+ 144us) client_negotiation.cc:240] Negotiation successful
0928 16:41:09.895183 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11588,"mutex_wait_us":77,"thread_start_us":10236,"threads_started":1}
I20260928 16:41:09.896543 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.878598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.10.247.193:44613)
0928 16:41:09.886297 (+ 7699us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.886307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.887199 (+ 892us) server_negotiation.cc:407] Connection header received
0928 16:41:09.887324 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.887333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.887632 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.887852 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:09.888664 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.890158 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.892661 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.894254 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.894461 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.894688 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:09.896012 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0928 16:41:09.896204 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7519,"thread_start_us":7357,"threads_started":1}
I20260928 16:41:09.896903 27846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.873959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36184 (local address 127.10.247.195:34147)
0928 16:41:09.885399 (+ 11440us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.885412 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.885462 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:41:09.885778 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.885791 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.886274 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.887383 (+ 1109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:09.887795 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.889407 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.892704 (+ 3297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.894254 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.896051 (+ 1797us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.896255 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:09.896338 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 16:41:09.896506 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11239,"thread_start_us":11049,"threads_started":1}
I20260928 16:41:09.896919 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.873508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34663 (local address 127.0.0.1:46828)
0928 16:41:09.884375 (+ 10867us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:09.884434 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:09.884747 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:09.888675 (+ 3928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:09.888680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:09.888706 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:09.889087 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.889099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.890899 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.890907 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.892968 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.892979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.894430 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.894436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.894704 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.894770 (+ 66us) client_negotiation.cc:769] Sending connection context
0928 16:41:09.894930 (+ 160us) client_negotiation.cc:240] Negotiation successful
0928 16:41:09.896517 (+ 1587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10503,"mutex_wait_us":1385,"thread_start_us":11986,"threads_started":1}
I20260928 16:41:09.901053 27635 tablet_service.cc:1467] Processing CreateTablet for tablet 64b3af9476d94ddf93c467f12e53d9ea (DEFAULT_TABLE table=test-table [id=7639fedf9257483ebd0ef73d92a3360f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:41:09.901059 27767 tablet_service.cc:1467] Processing CreateTablet for tablet 64b3af9476d94ddf93c467f12e53d9ea (DEFAULT_TABLE table=test-table [id=7639fedf9257483ebd0ef73d92a3360f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:41:09.901049 27503 tablet_service.cc:1467] Processing CreateTablet for tablet 64b3af9476d94ddf93c467f12e53d9ea (DEFAULT_TABLE table=test-table [id=7639fedf9257483ebd0ef73d92a3360f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:41:09.902784 27635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b3af9476d94ddf93c467f12e53d9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:09.902792 27503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b3af9476d94ddf93c467f12e53d9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:09.902805 27767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b3af9476d94ddf93c467f12e53d9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:09.918188 27850 tablet_bootstrap.cc:492] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: Bootstrap starting.
I20260928 16:41:09.920773 27851 tablet_bootstrap.cc:492] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: Bootstrap starting.
I20260928 16:41:09.923308 27850 tablet_bootstrap.cc:654] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:09.924362 27852 tablet_bootstrap.cc:492] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Bootstrap starting.
I20260928 16:41:09.924530 27851 tablet_bootstrap.cc:654] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:09.926100 27851 log.cc:824] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:09.926131 27850 log.cc:824] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:09.928292 27852 tablet_bootstrap.cc:654] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:09.929955 27852 log.cc:824] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:09.930094 27851 tablet_bootstrap.cc:492] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: No bootstrap required, opened a new log
I20260928 16:41:09.930121 27850 tablet_bootstrap.cc:492] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: No bootstrap required, opened a new log
I20260928 16:41:09.930446 27851 ts_tablet_manager.cc:1397] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260928 16:41:09.930466 27850 ts_tablet_manager.cc:1397] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260928 16:41:09.941277 27852 tablet_bootstrap.cc:492] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: No bootstrap required, opened a new log
I20260928 16:41:09.941721 27852 ts_tablet_manager.cc:1397] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.003s
I20260928 16:41:09.943289 27850 raft_consensus.cc:348] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:41:09.943912 27850 raft_consensus.cc:374] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:09.944078 27850 raft_consensus.cc:729] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c100c335d44c38b74794e652ac9863, State: Initialized, Role: FOLLOWER
I20260928 16:41:09.944146 27851 raft_consensus.cc:348] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:41:09.944662 27851 raft_consensus.cc:374] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:09.944777 27851 raft_consensus.cc:729] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2711cedc194aabb6a432d876c89c9f, State: Initialized, Role: FOLLOWER
I20260928 16:41:09.944964 27850 consensus_queue.cc:260] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:41:09.945536 27851 consensus_queue.cc:260] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:41:09.946475 27700 heartbeater.cc:503] Master 127.10.247.254:45277 was elected leader, sending a full tablet report...
I20260928 16:41:09.947399 27850 ts_tablet_manager.cc:1428] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: Time spent starting tablet: real 0.017s user 0.010s sys 0.004s
I20260928 16:41:09.948469 27832 heartbeater.cc:503] Master 127.10.247.254:45277 was elected leader, sending a full tablet report...
I20260928 16:41:09.949877 27851 ts_tablet_manager.cc:1428] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: Time spent starting tablet: real 0.019s user 0.002s sys 0.016s
I20260928 16:41:09.953543 27852 raft_consensus.cc:348] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:41:09.954051 27852 raft_consensus.cc:374] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:09.954176 27852 raft_consensus.cc:729] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fc5855301b41219e1d385a839506a0, State: Initialized, Role: FOLLOWER
I20260928 16:41:09.954870 27852 consensus_queue.cc:260] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:41:09.956511 27852 ts_tablet_manager.cc:1428] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
W20260928 16:41:09.959273 27569 tablet.cc:2369] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:41:09.974882 27857 raft_consensus.cc:482] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:41:09.975308 27857 raft_consensus.cc:504] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:41:09.977583 27857 leader_election.cc:288] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4c100c335d44c38b74794e652ac9863 (127.10.247.194:34663), 93fc5855301b41219e1d385a839506a0 (127.10.247.193:44613)
I20260928 16:41:09.985174 27835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.978477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34663 (local address 127.10.247.195:51277)
0928 16:41:09.978734 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:09.978751 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:09.978917 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:09.979398 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:09.979403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:09.979435 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:09.979649 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.979660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.980866 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.980872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.982599 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.982613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.984253 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.984259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.984467 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.984507 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 16:41:09.984655 (+ 148us) client_negotiation.cc:240] Negotiation successful
0928 16:41:09.984830 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 16:41:09.985162 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.978162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:51277 (local address 127.10.247.194:34663)
0928 16:41:09.978432 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.978439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.978843 (+ 404us) server_negotiation.cc:407] Connection header received
0928 16:41:09.979149 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.979156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.979253 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.979375 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:09.979784 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.980714 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.982809 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.984058 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.984202 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.984269 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:09.984692 (+ 423us) server_negotiation.cc:299] Negotiation successful
0928 16:41:09.984821 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 16:41:09.986227 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.978282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:47393 (local address 127.10.247.193:44613)
0928 16:41:09.978530 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:09.978535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:09.979530 (+ 995us) server_negotiation.cc:407] Connection header received
0928 16:41:09.979704 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:09.979709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:09.979792 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:09.979873 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:09.980566 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.981670 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.983852 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.985106 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.985275 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.985356 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:09.985798 (+ 442us) server_negotiation.cc:299] Negotiation successful
0928 16:41:09.985961 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260928 16:41:09.986822 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:09.978484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.10.247.195:47393)
0928 16:41:09.979405 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:09.979420 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:09.979551 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:09.979897 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:09.979900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:09.979923 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:09.980394 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.980409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.981825 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.981830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.983653 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:09.983670 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:09.985403 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:09.985409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:09.985595 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:09.985630 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 16:41:09.986276 (+ 646us) client_negotiation.cc:240] Negotiation successful
0928 16:41:09.986437 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":93,"thread_start_us":528,"threads_started":1}
I20260928 16:41:09.986909 27655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b3af9476d94ddf93c467f12e53d9ea" candidate_uuid: "5b2711cedc194aabb6a432d876c89c9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c100c335d44c38b74794e652ac9863" is_pre_election: true
I20260928 16:41:09.987643 27655 raft_consensus.cc:2454] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2711cedc194aabb6a432d876c89c9f in term 0.
I20260928 16:41:09.987950 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b3af9476d94ddf93c467f12e53d9ea" candidate_uuid: "5b2711cedc194aabb6a432d876c89c9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fc5855301b41219e1d385a839506a0" is_pre_election: true
I20260928 16:41:09.988482 27720 leader_election.cc:302] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2711cedc194aabb6a432d876c89c9f, d4c100c335d44c38b74794e652ac9863; no voters:
I20260928 16:41:09.988539 27523 raft_consensus.cc:2454] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2711cedc194aabb6a432d876c89c9f in term 0.
I20260928 16:41:09.988973 27857 raft_consensus.cc:2781] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:41:09.989145 27857 raft_consensus.cc:482] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:41:09.989274 27857 raft_consensus.cc:3037] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:09.991927 27857 raft_consensus.cc:504] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:41:09.992684 27857 leader_election.cc:288] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [CANDIDATE]: Term 1 election: Requested vote from peers d4c100c335d44c38b74794e652ac9863 (127.10.247.194:34663), 93fc5855301b41219e1d385a839506a0 (127.10.247.193:44613)
I20260928 16:41:09.993295 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b3af9476d94ddf93c467f12e53d9ea" candidate_uuid: "5b2711cedc194aabb6a432d876c89c9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fc5855301b41219e1d385a839506a0"
I20260928 16:41:09.993326 27655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b3af9476d94ddf93c467f12e53d9ea" candidate_uuid: "5b2711cedc194aabb6a432d876c89c9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c100c335d44c38b74794e652ac9863"
I20260928 16:41:09.993561 27655 raft_consensus.cc:3037] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:09.993551 27523 raft_consensus.cc:3037] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [term 0 FOLLOWER]: Advancing to term 1
W20260928 16:41:09.995319 27701 tablet.cc:2369] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:41:09.996335 27655 raft_consensus.cc:2454] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2711cedc194aabb6a432d876c89c9f in term 1.
I20260928 16:41:09.996840 27720 leader_election.cc:302] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2711cedc194aabb6a432d876c89c9f, d4c100c335d44c38b74794e652ac9863; no voters:
I20260928 16:41:09.996997 27523 raft_consensus.cc:2454] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2711cedc194aabb6a432d876c89c9f in term 1.
I20260928 16:41:09.997154 27857 raft_consensus.cc:2781] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:41:09.997743 27857 raft_consensus.cc:686] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 1 LEADER]: Becoming Leader. State: Replica: 5b2711cedc194aabb6a432d876c89c9f, State: Running, Role: LEADER
I20260928 16:41:09.998327 27857 consensus_queue.cc:237] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } }
I20260928 16:41:10.003916 27382 catalog_manager.cc:5495] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2711cedc194aabb6a432d876c89c9f (127.10.247.195). New cstate: current_term: 1 leader_uuid: "5b2711cedc194aabb6a432d876c89c9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b2711cedc194aabb6a432d876c89c9f" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 34147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 } health_report { overall_health: UNKNOWN } } }
I20260928 16:41:10.025453 27846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.019327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.10.247.195:34147)
0928 16:41:10.019565 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:10.019570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:10.019592 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:41:10.019727 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:10.019731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:10.019830 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:10.019978 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:41:10.020501 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:10.021359 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:10.022903 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:10.023346 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:10.023449 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:10.024033 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:41:10.024785 (+ 752us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:41:10.024917 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:10.025075 (+ 158us) server_negotiation.cc:299] Negotiation successful
0928 16:41:10.025199 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260928 16:41:10.026886 27833 tablet.cc:2369] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:41:10.036194 27655 raft_consensus.cc:1261] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [term 1 FOLLOWER]: Refusing update from remote peer 5b2711cedc194aabb6a432d876c89c9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:41:10.036194 27523 raft_consensus.cc:1261] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [term 1 FOLLOWER]: Refusing update from remote peer 5b2711cedc194aabb6a432d876c89c9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:41:10.037330 27860 consensus_queue.cc:1035] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c100c335d44c38b74794e652ac9863" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:41:10.037771 27857 consensus_queue.cc:1035] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fc5855301b41219e1d385a839506a0" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:41:10.055203 27865 mvcc.cc:205] Tried to move back new op lower bound from 7334353592455704576 to 7334353592317706240. Current Snapshot: MvccSnapshot[applied={T|T < 7334353592455704576}]
I20260928 16:41:10.065662 27381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:41:10.066026 27381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:41:10.070657 27381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Sending DeleteTablet for 3 replicas of tablet 64b3af9476d94ddf93c467f12e53d9ea
I20260928 16:41:10.072230 27635 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b3af9476d94ddf93c467f12e53d9ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:41:10 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:46828
I20260928 16:41:10.072357 27767 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b3af9476d94ddf93c467f12e53d9ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:41:10 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:36184
I20260928 16:41:10.072680 27503 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b3af9476d94ddf93c467f12e53d9ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:41:10 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:42532
I20260928 16:41:10.072898 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 27420
I20260928 16:41:10.084401 27874 tablet_replica.cc:330] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: stopping tablet replica
I20260928 16:41:10.085423 27874 raft_consensus.cc:2229] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:41:10.086387 27874 raft_consensus.cc:2258] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:41:10.089941 27874 ts_tablet_manager.cc:1905] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:41:10.090780 27873 tablet_replica.cc:330] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: stopping tablet replica
I20260928 16:41:10.091447 27873 raft_consensus.cc:2229] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:41:10.092233 27873 raft_consensus.cc:2258] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:41:10.093888 27872 tablet_replica.cc:330] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: stopping tablet replica
I20260928 16:41:10.094583 27872 raft_consensus.cc:2229] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:41:10.095530 27872 raft_consensus.cc:2258] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:41:10.096988 27873 ts_tablet_manager.cc:1905] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:41:10.098212 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.074049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42544 (local address 127.10.247.193:44613)
0928 16:41:10.074273 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:10.074279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:10.074299 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:41:10.074577 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:10.074584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:10.074684 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:10.074786 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:10.077221 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:10.078663 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:10.081035 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:10.081864 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:10.082001 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:10.086780 (+ 4779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:10.086804 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:10.086831 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:10.086900 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:10.091550 (+ 4650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:10.092695 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:10.092711 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:10.092715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:10.093162 (+ 447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:10.093698 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:10.093704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:10.093707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:10.094296 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:10.094635 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:10.097455 (+ 2820us) server_negotiation.cc:299] Negotiation successful
0928 16:41:10.097770 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 16:41:10.099138 27563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:10.099752 27872 ts_tablet_manager.cc:1905] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:41:10.105536 27873 ts_tablet_manager.cc:1918] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:10.106559 27873 log.cc:1196] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/64b3af9476d94ddf93c467f12e53d9ea
I20260928 16:41:10.107805 27872 ts_tablet_manager.cc:1918] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:10.108186 27872 log.cc:1196] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/64b3af9476d94ddf93c467f12e53d9ea
I20260928 16:41:10.108677 27873 ts_tablet_manager.cc:1939] T 64b3af9476d94ddf93c467f12e53d9ea P d4c100c335d44c38b74794e652ac9863: Deleting consensus metadata
I20260928 16:41:10.108836 27872 ts_tablet_manager.cc:1939] T 64b3af9476d94ddf93c467f12e53d9ea P 5b2711cedc194aabb6a432d876c89c9f: Deleting consensus metadata
I20260928 16:41:10.111491 27368 catalog_manager.cc:4841] TS 5b2711cedc194aabb6a432d876c89c9f (127.10.247.195:34147): tablet 64b3af9476d94ddf93c467f12e53d9ea (table test-table [id=7639fedf9257483ebd0ef73d92a3360f]) successfully deleted
I20260928 16:41:10.113581 27366 catalog_manager.cc:4841] TS d4c100c335d44c38b74794e652ac9863 (127.10.247.194:34663): tablet 64b3af9476d94ddf93c467f12e53d9ea (table test-table [id=7639fedf9257483ebd0ef73d92a3360f]) successfully deleted
I20260928 16:41:10.228816 27874 ts_tablet_manager.cc:1918] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:10.229161 27874 log.cc:1196] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/64b3af9476d94ddf93c467f12e53d9ea
I20260928 16:41:10.229789 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 27420
I20260928 16:41:10.229879 27874 ts_tablet_manager.cc:1939] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Deleting consensus metadata
I20260928 16:41:10.252239 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 27572
W20260928 16:41:10.252944 27369 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.247.193:44613 (error 108)
I20260928 16:41:10.256276 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.255656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:42550)
0928 16:41:10.255947 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:10.256071 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:44613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260928 16:41:10.256824 27369 catalog_manager.cc:4584] TS 93fc5855301b41219e1d385a839506a0 (127.10.247.193:44613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64b3af9476d94ddf93c467f12e53d9ea: Network error: Client connection negotiation failed: client connection to 127.10.247.193:44613: connect: Connection refused (error 111)
I20260928 16:41:10.274050 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.254897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.10.247.194:34663)
0928 16:41:10.255139 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:10.255146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:10.255168 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:41:10.255241 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:10.255247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:10.255334 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:10.255437 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:10.257599 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:10.258923 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:10.261366 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:10.262078 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:10.262220 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:10.266121 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:10.266143 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:10.266167 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:10.266217 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:10.270299 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:10.271236 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:10.271247 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:10.271251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:10.271430 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:10.271850 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:10.271855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:10.271859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:10.272439 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:10.272812 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:10.273377 (+ 565us) server_negotiation.cc:299] Negotiation successful
0928 16:41:10.273658 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:41:10.274922 27695 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:10.311384 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.310669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:42556)
0928 16:41:10.310961 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:10.311128 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:44613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260928 16:41:10.312888 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.312340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:42566)
0928 16:41:10.312524 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:10.312654 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:44613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260928 16:41:10.373154 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 27572
I20260928 16:41:10.376008 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.375354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:42570)
0928 16:41:10.375621 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:10.375748 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:44613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260928 16:41:10.377395 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.376913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:42586)
0928 16:41:10.377097 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:10.377205 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:44613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 16:41:10.391723 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 27704
I20260928 16:41:10.410558 27846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.392792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.10.247.195:34147)
0928 16:41:10.393017 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:10.393024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:10.393136 (+ 112us) server_negotiation.cc:407] Connection header received
0928 16:41:10.393371 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:10.393377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:10.393519 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:10.393638 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:10.395271 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:10.396246 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:10.398519 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:10.399046 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:10.399174 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:10.403262 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:10.403282 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:10.403304 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:10.403357 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:10.406827 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:10.407984 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:10.407994 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:10.407996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:10.408202 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:10.408734 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:10.408737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:10.408739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:10.409213 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:10.409490 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:10.409928 (+ 438us) server_negotiation.cc:299] Negotiation successful
0928 16:41:10.410162 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260928 16:41:10.411569 27827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:10.458468 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.457754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:42592)
0928 16:41:10.458071 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:10.458221 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:44613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260928 16:41:10.459954 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.459408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:42596)
0928 16:41:10.459668 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:10.459778 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:44613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260928 16:41:10.551149 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 27704
I20260928 16:41:10.571384 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 27349
I20260928 16:41:10.572821 27413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:10.705142 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 27349
I20260928 16:41:10.705212 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:10.704643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44613 (local address 127.0.0.1:42602)
0928 16:41:10.704881 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:10.704994 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:44613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260928 16:41:30.728473 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45277
--webserver_interface=127.10.247.254
--webserver_port=35587
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:45277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:41:31.128316 27880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:31.128791 27880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:31.128937 27880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:31.137259 27880 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 16:41:31.137408 27880 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 16:41:31.137465 27880 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 16:41:31.137542 27880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:41:31.137619 27880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:31.137722 27880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:31.137785 27880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:41:31.137862 27880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:41:31.149505 27880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:45277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:45277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=35587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:31.151360 27880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:31.153088 27880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:31.167436 27885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:31.167694 27880 server_base.cc:1034] running on GCE node
W20260928 16:41:31.167464 27886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:31.167698 27888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:31.169090 27880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:31.170951 27880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:31.172220 27880 hybrid_clock.cc:648] HybridClock initialized: now 1790613691172151 us; error 41 us; skew 500 ppm
I20260928 16:41:31.172647 27880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:31 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613691, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:31.176657 27880 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:41:31.178622 27880 webserver.cc:460] Webserver started at http://127.10.247.254:35587/ using document root <none> and password file <none>
I20260928 16:41:31.179556 27880 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:31.179662 27880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:31.187008 27880 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:41:31.190907 27895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:31.192750 27880 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:41:31.192960 27880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "725959fc0f6d4e2a8abfe10e14e86bff"
format_stamp: "Formatted at 2026-09-28 16:40:44 on dist-test-slave-9fn6"
server_key: "24156a6136f67ed377dd71fc689c6fd5"
server_key_iv: "e347223c21a9dd98325a01ecb552dafa"
server_key_version: "encryptionkey@0"
I20260928 16:41:31.194221 27880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:31.239130 27880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:31.245108 27880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:31.245496 27880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:31.268457 27880 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:45277
I20260928 16:41:31.268479 27946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:45277 every 8 connection(s)
I20260928 16:41:31.270582 27880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:41:31.272823 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 27880
I20260928 16:41:31.273864 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:44613
--local_ip_for_outbound_sockets=127.10.247.193
--tserver_master_addrs=127.10.247.254:45277
--webserver_port=44941
--webserver_interface=127.10.247.193
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:41:31.279758 27947 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:41:31.284071 27947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Bootstrap starting.
I20260928 16:41:31.313788 27947 log.cc:824] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:31.332774 27947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:41:31.333417 27947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Bootstrap complete.
I20260928 16:41:31.343508 27947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:41:31.344579 27947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725959fc0f6d4e2a8abfe10e14e86bff, State: Initialized, Role: FOLLOWER
I20260928 16:41:31.345386 27947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:41:31.345654 27947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:41:31.345789 27947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:41:31.345971 27947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 2 FOLLOWER]: Advancing to term 3
I20260928 16:41:31.350337 27947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:41:31.350893 27947 leader_election.cc:302] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 725959fc0f6d4e2a8abfe10e14e86bff; no voters:
I20260928 16:41:31.351632 27947 leader_election.cc:288] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 16:41:31.351866 27951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 3 FOLLOWER]: Leader election won for term 3
I20260928 16:41:31.352859 27951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [term 3 LEADER]: Becoming Leader. State: Replica: 725959fc0f6d4e2a8abfe10e14e86bff, State: Running, Role: LEADER
I20260928 16:41:31.353575 27951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } }
I20260928 16:41:31.353816 27947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:41:31.356598 27952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } } }
I20260928 16:41:31.357132 27952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:31.356667 27953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 725959fc0f6d4e2a8abfe10e14e86bff. Latest consensus state: current_term: 3 leader_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725959fc0f6d4e2a8abfe10e14e86bff" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 45277 } } }
I20260928 16:41:31.357824 27953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:31.362870 27958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:41:31.373958 27958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:41:31.376008 27958 catalog_manager.cc:1259] Loaded cluster ID: e50fb48416594b4fb03ca3ef68b484ea
I20260928 16:41:31.376288 27958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:41:31.381598 27958 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:41:31.383821 27958 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Loaded TSK: 0
I20260928 16:41:31.384739 27958 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 725959fc0f6d4e2a8abfe10e14e86bff: Loaded TSK: 1
F20260928 16:41:31.386116 27958 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb13e37e9ad google::LogMessage::Fail() at ??:0
@ 0x7fb13e38188a google::LogMessage::SendToLog() at ??:0
@ 0x7fb13e37e4de google::LogMessage::Flush() at ??:0
@ 0x7fb13e37fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb1420e27d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb1420d173e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb153dad1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb153dac783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb153e382a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fb153e380c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fb153e825e9 std::function<>::operator()() at ??:0
@ 0x7fb153db0cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb153daec36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb153e36610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb153e36439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb140fca345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb140fcd91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb140fcd799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb140fa8eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb1442826db start_thread at ??:0
@ 0x7fb13d21861f clone at ??:0
W20260928 16:41:31.707893 27949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:31.708273 27949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:31.708387 27949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:31.716509 27949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:31.716696 27949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:31.716867 27949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:41:31.728199 27949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:44613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=44941
--tserver_master_addrs=127.10.247.254:45277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:31.730183 27949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:31.731858 27949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:31.747680 27994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:31.747656 27995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:31.747601 27997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:31.749166 27949 server_base.cc:1034] running on GCE node
I20260928 16:41:31.749940 27949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:31.751153 27949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:31.752405 27949 hybrid_clock.cc:648] HybridClock initialized: now 1790613691752337 us; error 54 us; skew 500 ppm
I20260928 16:41:31.752838 27949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:31 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613691, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:31.756768 27949 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:41:31.758682 27949 webserver.cc:460] Webserver started at http://127.10.247.193:44941/ using document root <none> and password file <none>
I20260928 16:41:31.759522 27949 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:31.759639 27949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:31.766592 27949 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 16:41:31.770090 28004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:31.771735 27949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:41:31.771934 27949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "93fc5855301b41219e1d385a839506a0"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "3014e251e53142a12de2d8cb9389f8d1"
server_key_iv: "df8e2df5dc94be016ceeb84d3a634fef"
server_key_version: "encryptionkey@0"
I20260928 16:41:31.773294 27949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:31.795317 27949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:31.800549 27949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:31.800907 27949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:31.802213 27949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:31.804493 28011 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260928 16:41:31.811733 28011 ts_tablet_manager.cc:1852] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260928 16:41:31.813107 28011 ts_tablet_manager.cc:1905] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:41:31.823097 28011 ts_tablet_manager.cc:1918] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:31.823486 28011 ts_tablet_manager.cc:1939] T 64b3af9476d94ddf93c467f12e53d9ea P 93fc5855301b41219e1d385a839506a0: Deleting consensus metadata
I20260928 16:41:31.824092 27949 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260928 16:41:31.824237 27949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.020s user 0.001s sys 0.000s
I20260928 16:41:31.824404 27949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:31.824483 27949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:31.860739 27949 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:44613
I20260928 16:41:31.860801 28117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:44613 every 8 connection(s)
I20260928 16:41:31.862356 27949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:41:31.868366 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:31.866085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:46451)
0928 16:41:31.866973 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:31.867283 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":343,"threads_started":1}
I20260928 16:41:31.868957 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 27949
W20260928 16:41:31.869045 28118 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:45277 (0 consecutive failures): Network error: Failed to ping master at 127.10.247.254:45277: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
I20260928 16:41:31.869830 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:34663
--local_ip_for_outbound_sockets=127.10.247.194
--tserver_master_addrs=127.10.247.254:45277
--webserver_port=42175
--webserver_interface=127.10.247.194
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:41:31.870131 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:31.869719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:43101)
0928 16:41:31.869889 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:31.869975 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260928 16:41:31.871052 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:31.870707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:59425)
0928 16:41:31.870843 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:31.870921 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260928 16:41:31.871302 28118 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 16:41:32.281020 28121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:32.281411 28121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:32.281541 28121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:32.289570 28121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:32.289716 28121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:32.289853 28121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:41:32.300781 28121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:34663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=42175
--tserver_master_addrs=127.10.247.254:45277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:32.302685 28121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:32.304294 28121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:32.319629 28126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:32.319526 28129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:32.319566 28127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:32.320343 28121 server_base.cc:1034] running on GCE node
I20260928 16:41:32.321171 28121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:32.322410 28121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:32.323668 28121 hybrid_clock.cc:648] HybridClock initialized: now 1790613692323602 us; error 57 us; skew 500 ppm
I20260928 16:41:32.324048 28121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:32 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613692, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:32.327910 28121 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:41:32.329761 28121 webserver.cc:460] Webserver started at http://127.10.247.194:42175/ using document root <none> and password file <none>
I20260928 16:41:32.330662 28121 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:32.330776 28121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:32.337375 28121 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:41:32.340991 28136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:32.342900 28121 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:41:32.343114 28121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "d4c100c335d44c38b74794e652ac9863"
format_stamp: "Formatted at 2026-09-28 16:40:45 on dist-test-slave-9fn6"
server_key: "28424b679a627ace179804b8718174f3"
server_key_iv: "53cc8f6548dfe7d8306a9a78b8a234fa"
server_key_version: "encryptionkey@0"
I20260928 16:41:32.344370 28121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:32.362432 28121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:32.367635 28121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:32.367973 28121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:32.369246 28121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:32.370916 28121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:32.371011 28121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:32.371140 28121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:32.371232 28121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:32.405849 28121 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:34663
I20260928 16:41:32.405898 28248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:34663 every 8 connection(s)
I20260928 16:41:32.407661 28121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:41:32.413743 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:32.411390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:41585)
0928 16:41:32.412263 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:32.412575 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":326,"threads_started":1}
I20260928 16:41:32.414093 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 28121
W20260928 16:41:32.414439 28249 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:45277 (0 consecutive failures): Network error: Failed to ping master at 127.10.247.254:45277: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
I20260928 16:41:32.414918 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:34147
--local_ip_for_outbound_sockets=127.10.247.195
--tserver_master_addrs=127.10.247.254:45277
--webserver_port=39775
--webserver_interface=127.10.247.195
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:41:32.415578 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:32.415175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:44121)
0928 16:41:32.415345 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:32.415428 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 16:41:32.416568 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:32.416136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:60611)
0928 16:41:32.416308 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:32.416411 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260928 16:41:32.416926 28249 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 16:41:32.808365 28252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:32.808782 28252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:32.808905 28252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:32.816929 28252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:32.817090 28252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:32.817226 28252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:41:32.828197 28252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:34147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=39775
--tserver_master_addrs=127.10.247.254:45277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:32.829955 28252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:32.831670 28252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:32.846552 28258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:32.846674 28257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:32.846590 28260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:32.847585 28252 server_base.cc:1034] running on GCE node
I20260928 16:41:32.848340 28252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:32.849557 28252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:32.850769 28252 hybrid_clock.cc:648] HybridClock initialized: now 1790613692850701 us; error 56 us; skew 500 ppm
I20260928 16:41:32.851161 28252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:32 dist-test-slave-9fn6 krb5kdc[26794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613692, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:32.854701 28252 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:41:32.856490 28252 webserver.cc:460] Webserver started at http://127.10.247.195:39775/ using document root <none> and password file <none>
I20260928 16:41:32.857412 28252 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:32.857545 28252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:32.864113 28252 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260928 16:41:32.867465 28267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:32.869264 28252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260928 16:41:32.869444 28252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "5b2711cedc194aabb6a432d876c89c9f"
format_stamp: "Formatted at 2026-09-28 16:40:46 on dist-test-slave-9fn6"
server_key: "f25b3d9b9acaa2bb4f14d8597c01b77b"
server_key_iv: "59b201e4f2d90a635875000a2a014a1a"
server_key_version: "encryptionkey@0"
I20260928 16:41:32.870638 28252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:32.873040 28268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:32.872121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:57849)
0928 16:41:32.872658 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:32.872775 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":247,"threads_started":1}
I20260928 16:41:32.888672 28252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:32.893791 28252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:32.894109 28252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:32.895350 28252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:32.897089 28252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:32.897200 28252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:32.897316 28252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:32.897390 28252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:32.932436 28252 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:34147
I20260928 16:41:32.932520 28380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:34147 every 8 connection(s)
I20260928 16:41:32.934278 28252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:41:32.937829 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 28252
I20260928 16:41:32.940315 28383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:32.937875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:39419)
0928 16:41:32.938795 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:32.939067 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":330,"threads_started":1}
W20260928 16:41:32.941032 28381 heartbeater.cc:644] Failed to heartbeat to 127.10.247.254:45277 (0 consecutive failures): Network error: Failed to ping master at 127.10.247.254:45277: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
I20260928 16:41:32.941915 28383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:32.941576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:35271)
0928 16:41:32.941723 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:32.941796 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 16:41:32.942704 28383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:32.942409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:51087)
0928 16:41:32.942517 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:32.942588 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 16:41:32.942901 28381 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 16:41:33.418754 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:33.417723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:43049)
0928 16:41:33.418325 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:33.418466 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":268,"threads_started":1}
I20260928 16:41:33.875170 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:33.874190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:57361)
0928 16:41:33.874811 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:33.874938 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":278,"threads_started":1}
I20260928 16:41:33.944643 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:33.943582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:59479)
0928 16:41:33.944179 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:33.944324 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":286,"threads_started":1}
I20260928 16:41:34.420796 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:34.419871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:52901)
0928 16:41:34.420418 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:34.420523 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":265,"threads_started":1}
I20260928 16:41:34.877489 28389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:34.876435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:48419)
0928 16:41:34.877106 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:34.877233 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":362,"threads_started":1}
I20260928 16:41:34.946941 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:34.945969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:42561)
0928 16:41:34.946598 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:34.946709 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":309,"threads_started":1}
I20260928 16:41:35.422899 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:35.421925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:51435)
0928 16:41:35.422563 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:35.422670 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":343,"threads_started":1}
I20260928 16:41:35.879655 28392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:35.878614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:49539)
0928 16:41:35.879296 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:35.879408 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":338,"threads_started":1}
I20260928 16:41:35.949153 28393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:35.948225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:40139)
0928 16:41:35.948821 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:35.948927 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":302,"threads_started":1}
I20260928 16:41:36.424934 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:36.423963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:52945)
0928 16:41:36.424543 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:36.424694 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":283,"threads_started":1}
I20260928 16:41:36.881671 28395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:36.880810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:38037)
0928 16:41:36.881363 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:36.881469 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":285,"threads_started":1}
I20260928 16:41:36.951184 28396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:36.950275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:44857)
0928 16:41:36.950830 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:36.950935 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":266,"threads_started":1}
I20260928 16:41:37.427277 28397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:37.426281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:49625)
0928 16:41:37.426941 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:37.427051 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":342,"threads_started":1}
I20260928 16:41:37.883740 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:37.882823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:44009)
0928 16:41:37.883400 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:37.883503 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":327,"threads_started":1}
I20260928 16:41:37.953226 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:37.952289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:55125)
0928 16:41:37.952904 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:37.953011 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":332,"threads_started":1}
I20260928 16:41:38.429539 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:38.428463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:48325)
0928 16:41:38.429143 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:38.429297 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":340,"threads_started":1}
I20260928 16:41:38.885938 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:38.884873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:50893)
0928 16:41:38.885599 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:38.885695 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":309,"threads_started":1}
I20260928 16:41:38.955159 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:38.954255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:37467)
0928 16:41:38.954826 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:38.954936 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":292,"threads_started":1}
I20260928 16:41:39.431638 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:39.430644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:56619)
0928 16:41:39.431273 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:39.431375 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":244,"threads_started":1}
I20260928 16:41:39.888019 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:39.887051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:37199)
0928 16:41:39.887679 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:39.887789 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":300,"threads_started":1}
I20260928 16:41:39.957453 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:39.956384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:40575)
0928 16:41:39.957050 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:39.957179 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":306,"threads_started":1}
I20260928 16:41:40.433826 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:40.432890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:38213)
0928 16:41:40.433512 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:40.433613 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":349,"threads_started":1}
I20260928 16:41:40.890447 28407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:40.889448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:34873)
0928 16:41:40.890096 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:40.890233 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":312,"threads_started":1}
I20260928 16:41:40.959712 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:40.958758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:54453)
0928 16:41:40.959349 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:40.959483 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":292,"threads_started":1}
I20260928 16:41:41.435972 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:41.435030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:37939)
0928 16:41:41.435634 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:41.435744 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":289,"threads_started":1}
I20260928 16:41:41.892776 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:41.891798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:34955)
0928 16:41:41.892372 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:41.892474 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":302,"threads_started":1}
I20260928 16:41:41.961905 28411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:41.960905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:57375)
0928 16:41:41.961546 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:41.961661 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":314,"threads_started":1}
I20260928 16:41:42.438205 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:42.437251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:43533)
0928 16:41:42.437844 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:42.437962 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":307,"threads_started":1}
I20260928 16:41:42.895044 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:42.893996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:33215)
0928 16:41:42.894646 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:42.894757 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":346,"threads_started":1}
I20260928 16:41:42.964083 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:42.963076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:50515)
0928 16:41:42.963706 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:42.963836 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":309,"threads_started":1}
I20260928 16:41:43.440315 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:43.439360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:38505)
0928 16:41:43.439932 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:43.440043 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":302,"threads_started":1}
I20260928 16:41:43.897383 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:43.896417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:54773)
0928 16:41:43.897038 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:43.897156 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":338,"threads_started":1}
I20260928 16:41:43.966256 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:43.965318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:60239)
0928 16:41:43.965868 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:43.965986 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":270,"threads_started":1}
I20260928 16:41:44.442566 28418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:44.441580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:59389)
0928 16:41:44.442204 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:44.442333 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":304,"threads_started":1}
I20260928 16:41:44.899554 28419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:44.898588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:33393)
0928 16:41:44.899191 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:44.899334 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":290,"threads_started":1}
I20260928 16:41:44.968353 28420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:44.967429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:52213)
0928 16:41:44.968029 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:44.968139 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":302,"threads_started":1}
I20260928 16:41:45.444736 28421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:45.443696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:43225)
0928 16:41:45.444267 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:45.444427 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":271,"threads_started":1}
I20260928 16:41:45.901932 28422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:45.900914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:38199)
0928 16:41:45.901553 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:45.901674 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":308,"threads_started":1}
I20260928 16:41:45.970713 28423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:45.969700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:41035)
0928 16:41:45.970346 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:45.970472 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":298,"threads_started":1}
I20260928 16:41:46.446827 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:46.445923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:34315)
0928 16:41:46.446458 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:46.446586 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":271,"threads_started":1}
I20260928 16:41:46.904589 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:46.903584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:60179)
0928 16:41:46.904238 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:46.904371 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":269,"threads_started":1}
I20260928 16:41:46.972713 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:46.971807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:55605)
0928 16:41:46.972358 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:46.972455 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":285,"threads_started":1}
I20260928 16:41:47.449270 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:47.448253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.194:54417)
0928 16:41:47.448866 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:47.448992 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":298,"threads_started":1}
I20260928 16:41:47.907003 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:47.906037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.193:42873)
0928 16:41:47.906672 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:47.906782 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":325,"threads_started":1}
I20260928 16:41:47.946143 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 27949
I20260928 16:41:47.974257 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:47.947699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.10.247.193:44613)
0928 16:41:47.948427 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:47.948446 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:47.948498 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:41:47.950744 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:47.950758 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:47.951363 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:47.951744 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:47.953536 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:47.955072 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:47.956549 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:47.957411 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:47.957565 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:47.966262 (+ 8697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:47.966274 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:47.966299 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:47.966371 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:47.969657 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:47.970616 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:47.970627 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:47.970637 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:47.970878 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:47.971544 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:47.971549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:47.971551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:47.972363 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:47.972736 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:47.973224 (+ 488us) server_negotiation.cc:299] Negotiation successful
0928 16:41:47.973780 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":302,"threads_started":1}
I20260928 16:41:47.975165 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:47.974158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:45277 (local address 127.10.247.195:57959)
0928 16:41:47.974781 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:47.974901 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.254:45277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"threads_started":1}
I20260928 16:41:47.976184 28113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:48.071394 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 27949
I20260928 16:41:48.091339 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 28121
I20260928 16:41:48.113438 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:48.092774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46162 (local address 127.10.247.194:34663)
0928 16:41:48.093519 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:48.093537 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:48.093589 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:41:48.095479 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:48.095492 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:48.096084 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:48.096475 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:48.098352 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:48.099832 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:48.101439 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:48.102336 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:48.102513 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:48.105674 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:48.105684 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:48.105708 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:48.105763 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:48.109297 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:48.110261 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:48.110271 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:48.110281 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:48.110528 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:48.110935 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:48.110939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:48.110941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:48.111688 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:48.111983 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:48.112453 (+ 470us) server_negotiation.cc:299] Negotiation successful
0928 16:41:48.113026 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":382,"threads_started":1}
I20260928 16:41:48.115331 28244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:48.207324 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 28121
I20260928 16:41:48.225207 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 28252
I20260928 16:41:48.246515 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:48.226551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.10.247.195:34147)
0928 16:41:48.227278 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:48.227296 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:48.227347 (+ 51us) server_negotiation.cc:407] Connection header received
0928 16:41:48.229365 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:48.229379 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:48.229968 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:48.230384 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:48.232122 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:48.233606 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:48.235081 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:48.235902 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:48.236130 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:48.239182 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:48.239192 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:48.239212 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:48.239265 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:48.242463 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:48.243298 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:48.243307 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:48.243316 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:48.243525 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:48.243954 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:48.243958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:48.243961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:48.244711 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:48.245042 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:48.245610 (+ 568us) server_negotiation.cc:299] Negotiation successful
0928 16:41:48.246158 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":340,"threads_started":1}
I20260928 16:41:48.248291 28376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:48.341228 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 28252
2026-09-28T16:41:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65456 ms)
[----------] 30 tests from SecurityITest (319787 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:41:48 dist-test-slave-9fn6 krb5kdc[28438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:41:48 dist-test-slave-9fn6 krb5kdc[28438](info): set up 2 sockets
Sep 28 16:41:48 dist-test-slave-9fn6 krb5kdc[28438](info): commencing operation
krb5kdc: starting...
W20260928 16:41:49.409113 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:41:49 dist-test-slave-9fn6 krb5kdc[28438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:41:49.737363 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:41741
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:41741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:41:50.147573 28451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:50.147969 28451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:50.148120 28451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:50.156101 28451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:41:50.156263 28451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:50.156325 28451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:50.156412 28451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:41:50.156472 28451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:41:50.167896 28451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:41741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:41741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:50.169795 28451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:50.171590 28451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:50.187052 28459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:50.187079 28457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:50.187121 28456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:50.188082 28451 server_base.cc:1034] running on GCE node
I20260928 16:41:50.189265 28451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:50.191304 28451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:50.192572 28451 hybrid_clock.cc:648] HybridClock initialized: now 1790613710192512 us; error 53 us; skew 500 ppm
I20260928 16:41:50.193091 28451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:50 dist-test-slave-9fn6 krb5kdc[28438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613710, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:50.197045 28451 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:41:50.199110 28451 webserver.cc:460] Webserver started at http://127.10.247.254:36695/ using document root <none> and password file <none>
I20260928 16:41:50.200029 28451 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:50.200184 28451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:50.200554 28451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:50.204650 28451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "949664e70cca4b359055b9c7250cf59d"
format_stamp: "Formatted at 2026-09-28 16:41:50 on dist-test-slave-9fn6"
server_key: "a1646053d4667ba7e9295dba008d2d83"
server_key_iv: "b1a72f4901820a5247ceff8c1c8494e9"
server_key_version: "encryptionkey@0"
I20260928 16:41:50.205681 28451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "949664e70cca4b359055b9c7250cf59d"
format_stamp: "Formatted at 2026-09-28 16:41:50 on dist-test-slave-9fn6"
server_key: "a1646053d4667ba7e9295dba008d2d83"
server_key_iv: "b1a72f4901820a5247ceff8c1c8494e9"
server_key_version: "encryptionkey@0"
I20260928 16:41:50.212805 28451 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260928 16:41:50.217540 28466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:50.219396 28451 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:41:50.219612 28451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "949664e70cca4b359055b9c7250cf59d"
format_stamp: "Formatted at 2026-09-28 16:41:50 on dist-test-slave-9fn6"
server_key: "a1646053d4667ba7e9295dba008d2d83"
server_key_iv: "b1a72f4901820a5247ceff8c1c8494e9"
server_key_version: "encryptionkey@0"
I20260928 16:41:50.219866 28451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:50.263541 28451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:50.269114 28451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:50.269443 28451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:50.290412 28451 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:41741
I20260928 16:41:50.290422 28517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:41741 every 8 connection(s)
I20260928 16:41:50.292104 28451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:41:50.297171 28518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:50.302521 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 28451
I20260928 16:41:50.302788 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:41:50.303732 11231 external_mini_cluster.cc:1428] Setting key 8b4e4a79fe4c518dc30377902aa707a9
I20260928 16:41:50.311400 28518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d: Bootstrap starting.
I20260928 16:41:50.315999 28518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:50.318087 28518 log.cc:824] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:50.322711 28518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d: No bootstrap required, opened a new log
Sep 28 16:41:50 dist-test-slave-9fn6 krb5kdc[28438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:41:50.335649 28518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949664e70cca4b359055b9c7250cf59d" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41741 } }
I20260928 16:41:50.336265 28518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:50.336395 28518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 949664e70cca4b359055b9c7250cf59d, State: Initialized, Role: FOLLOWER
I20260928 16:41:50.336555 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:50.306380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.10.247.254:41741)
0928 16:41:50.307328 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:50.307342 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:50.307392 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:41:50.309541 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:50.309572 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:50.310072 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:50.310452 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:50.312521 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:50.314085 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:50.316499 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:50.317951 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:50.318196 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:50.324545 (+ 6349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:50.324559 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:50.324589 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:50.324685 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:50.329719 (+ 5034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:50.330923 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:50.330936 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:50.330949 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:50.331243 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:50.331768 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:50.331773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:50.331776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:50.332831 (+ 1055us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:50.333205 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:50.333987 (+ 782us) server_negotiation.cc:299] Negotiation successful
0928 16:41:50.334782 (+ 795us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":441,"threads_started":1}
I20260928 16:41:50.337447 28518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949664e70cca4b359055b9c7250cf59d" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41741 } }
I20260928 16:41:50.337837 28518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:41:50.338006 28518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:41:50.338217 28518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:50.342043 28518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949664e70cca4b359055b9c7250cf59d" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41741 } }
I20260928 16:41:50.342768 28518 leader_election.cc:302] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 949664e70cca4b359055b9c7250cf59d; no voters:
I20260928 16:41:50.343515 28518 leader_election.cc:288] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:41:50.343604 28523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:41:50.344308 28523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [term 1 LEADER]: Becoming Leader. State: Replica: 949664e70cca4b359055b9c7250cf59d, State: Running, Role: LEADER
I20260928 16:41:50.345001 28523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949664e70cca4b359055b9c7250cf59d" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41741 } }
I20260928 16:41:50.346341 28518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:41:50.348685 28524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "949664e70cca4b359055b9c7250cf59d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949664e70cca4b359055b9c7250cf59d" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41741 } } }
I20260928 16:41:50.349099 28524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:50.348552 28525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 949664e70cca4b359055b9c7250cf59d. Latest consensus state: current_term: 1 leader_uuid: "949664e70cca4b359055b9c7250cf59d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949664e70cca4b359055b9c7250cf59d" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41741 } } }
I20260928 16:41:50.349589 28525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:50.351429 28529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:41:50.373962 28529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:41:50.388684 28529 catalog_manager.cc:1347] Generated new cluster ID: e24cf5dae6574e3c8e73dc2f2fe11720
I20260928 16:41:50.388852 28529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:41:50.404584 28529 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:41:50.406262 28529 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:41:50.414525 28529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d: Generated new TSK 0
I20260928 16:41:50.415755 28529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:41:50.493455 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:41741
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:41:50.894169 28544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:50.894544 28544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:50.894683 28544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:50.902971 28544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:50.903128 28544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:50.903297 28544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:41:50.914674 28544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:41741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:50.916450 28544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:50.918277 28544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:50.933759 28549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:50.933826 28552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:50.933797 28550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:50.934671 28544 server_base.cc:1034] running on GCE node
I20260928 16:41:50.935480 28544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:50.936759 28544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:50.937973 28544 hybrid_clock.cc:648] HybridClock initialized: now 1790613710937903 us; error 66 us; skew 500 ppm
I20260928 16:41:50.938344 28544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:50 dist-test-slave-9fn6 krb5kdc[28438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613710, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:50.941998 28544 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:41:50.943801 28544 webserver.cc:460] Webserver started at http://127.10.247.193:38425/ using document root <none> and password file <none>
I20260928 16:41:50.944675 28544 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:50.944808 28544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:50.945154 28544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:50.948673 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "06c76dd10a544f05a3c84f3ff7c47e00"
format_stamp: "Formatted at 2026-09-28 16:41:50 on dist-test-slave-9fn6"
server_key: "a975900868f99b8bed0797a534bb2a5a"
server_key_iv: "c311074bb617fe773e3ad86fc7f65175"
server_key_version: "encryptionkey@0"
I20260928 16:41:50.949518 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "06c76dd10a544f05a3c84f3ff7c47e00"
format_stamp: "Formatted at 2026-09-28 16:41:50 on dist-test-slave-9fn6"
server_key: "a975900868f99b8bed0797a534bb2a5a"
server_key_iv: "c311074bb617fe773e3ad86fc7f65175"
server_key_version: "encryptionkey@0"
I20260928 16:41:50.955900 28544 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:41:50.960328 28559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:50.962185 28544 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:41:50.962387 28544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "06c76dd10a544f05a3c84f3ff7c47e00"
format_stamp: "Formatted at 2026-09-28 16:41:50 on dist-test-slave-9fn6"
server_key: "a975900868f99b8bed0797a534bb2a5a"
server_key_iv: "c311074bb617fe773e3ad86fc7f65175"
server_key_version: "encryptionkey@0"
I20260928 16:41:50.962627 28544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:50.978868 28544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:50.984047 28544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:50.984376 28544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:50.985631 28544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:50.987298 28544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:50.987386 28544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:50.987530 28544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:50.987592 28544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:51.023494 28544 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:44451
I20260928 16:41:51.023530 28671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:44451 every 8 connection(s)
I20260928 16:41:51.025350 28544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:41:51.032188 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 28544
I20260928 16:41:51.032450 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:41:51.033195 11231 external_mini_cluster.cc:1428] Setting key 835fba2242d3b1a1c72dbd8f1e910070
Sep 28 16:41:51 dist-test-slave-9fn6 krb5kdc[28438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613710, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:41:51.051333 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:51.028398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:60045 (local address 127.10.247.254:41741)
0928 16:41:51.028954 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:51.028959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:51.030113 (+ 1154us) server_negotiation.cc:407] Connection header received
0928 16:41:51.032416 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:51.032422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:51.032527 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:51.032683 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:51.034472 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:51.035426 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:51.037853 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:51.038392 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:51.038549 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:51.043723 (+ 5174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:51.043731 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:51.043735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:51.043785 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:51.046957 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:51.048168 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:51.048176 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:51.048180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:51.048375 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:51.049076 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:51.049081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:51.049084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:51.049501 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:51.049676 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:51.050626 (+ 950us) server_negotiation.cc:299] Negotiation successful
0928 16:41:51.050906 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
I20260928 16:41:51.052513 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:51.029140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:41741 (local address 127.10.247.193:60045)
0928 16:41:51.029929 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:51.029971 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:51.032049 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:51.032965 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:51.032975 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:51.033610 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:51.034306 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:51.034334 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:51.035575 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:51.035581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:51.037653 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:51.037674 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:51.038669 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:51.038675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:51.039003 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:51.039902 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:41:51.039940 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:51.043319 (+ 3379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:51.047181 (+ 3862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:51.047195 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:51.047221 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:51.047980 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:51.048590 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:51.048595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:51.048597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:51.048917 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:51.049657 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:51.049666 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:51.050210 (+ 544us) client_negotiation.cc:769] Sending connection context
0928 16:41:51.050498 (+ 288us) client_negotiation.cc:240] Negotiation successful
0928 16:41:51.051046 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":295,"threads_started":1}
I20260928 16:41:51.054531 28672 heartbeater.cc:347] Connected to a master server at 127.10.247.254:41741
I20260928 16:41:51.055048 28672 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:51.056135 28672 heartbeater.cc:511] Master 127.10.247.254:41741 requested a full tablet report, sending...
I20260928 16:41:51.059211 28483 ts_manager.cc:194] Registered new tserver with Master: 06c76dd10a544f05a3c84f3ff7c47e00 (127.10.247.193:44451)
I20260928 16:41:51.061203 28483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:60045
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:41:51.108873 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:41741
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:41:51.507287 28678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:51.507704 28678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:51.507877 28678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:51.515924 28678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:51.516086 28678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:51.516284 28678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:41:51.527664 28678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:41741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:51.529651 28678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:51.531330 28678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:51.546674 28683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:51.546694 28686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:51.546770 28684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:51.547420 28678 server_base.cc:1034] running on GCE node
I20260928 16:41:51.548218 28678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:51.549465 28678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:51.550678 28678 hybrid_clock.cc:648] HybridClock initialized: now 1790613711550619 us; error 51 us; skew 500 ppm
I20260928 16:41:51.551050 28678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:51 dist-test-slave-9fn6 krb5kdc[28438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613711, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:51.554877 28678 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:41:51.556770 28678 webserver.cc:460] Webserver started at http://127.10.247.194:37377/ using document root <none> and password file <none>
I20260928 16:41:51.557623 28678 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:51.557735 28678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:51.558050 28678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:51.561769 28678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "7a4f7dde68dd4456bf6fab497d69ee05"
format_stamp: "Formatted at 2026-09-28 16:41:51 on dist-test-slave-9fn6"
server_key: "cfdb0b2bacb3796fc750245351c99612"
server_key_iv: "74ed37e7ed4ee06a033f104ba02c0860"
server_key_version: "encryptionkey@0"
I20260928 16:41:51.562680 28678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "7a4f7dde68dd4456bf6fab497d69ee05"
format_stamp: "Formatted at 2026-09-28 16:41:51 on dist-test-slave-9fn6"
server_key: "cfdb0b2bacb3796fc750245351c99612"
server_key_iv: "74ed37e7ed4ee06a033f104ba02c0860"
server_key_version: "encryptionkey@0"
I20260928 16:41:51.569409 28678 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:41:51.573805 28693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:51.575704 28678 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260928 16:41:51.575917 28678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "7a4f7dde68dd4456bf6fab497d69ee05"
format_stamp: "Formatted at 2026-09-28 16:41:51 on dist-test-slave-9fn6"
server_key: "cfdb0b2bacb3796fc750245351c99612"
server_key_iv: "74ed37e7ed4ee06a033f104ba02c0860"
server_key_version: "encryptionkey@0"
I20260928 16:41:51.576159 28678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:51.608840 28678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:51.614075 28678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:51.614404 28678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:51.615638 28678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:51.617395 28678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:51.617486 28678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:51.617621 28678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:51.617698 28678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:51.653165 28678 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:34315
I20260928 16:41:51.653226 28805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:34315 every 8 connection(s)
I20260928 16:41:51.654904 28678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:41:51.663761 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 28678
I20260928 16:41:51.663960 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:41:51.664541 11231 external_mini_cluster.cc:1428] Setting key e5f1210186995345ed7a0e797be3bc38
Sep 28 16:41:51 dist-test-slave-9fn6 krb5kdc[28438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613711, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:41:51.678856 28808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:51.658096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:58605 (local address 127.10.247.254:41741)
0928 16:41:51.658625 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:51.658630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:51.659669 (+ 1039us) server_negotiation.cc:407] Connection header received
0928 16:41:51.661920 (+ 2251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:51.661927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:51.662031 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:51.662145 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:51.664081 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:51.664990 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:51.666990 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:51.667729 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:51.667871 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:51.672196 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:51.672203 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:51.672207 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:51.672254 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:51.675128 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:51.676117 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:51.676122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:51.676124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:51.676274 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:51.676803 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:51.676806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:51.676808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:51.677142 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:51.677267 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:51.678278 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0928 16:41:51.678534 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":268,"threads_started":1}
I20260928 16:41:51.680263 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:51.658678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:41741 (local address 127.10.247.194:58605)
0928 16:41:51.659477 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:51.659532 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:51.661590 (+ 2058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:51.662456 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:51.662466 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:51.663116 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:51.663856 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:51.663883 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:51.665175 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:51.665180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:51.666822 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:51.666840 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:51.667991 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:51.667996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:51.668265 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:51.669146 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:41:51.669178 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:51.671887 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:51.675296 (+ 3409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:51.675306 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:51.675331 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:51.675978 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:51.676376 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:51.676380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:51.676384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:51.676697 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:51.677254 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:51.677264 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:51.677812 (+ 548us) client_negotiation.cc:769] Sending connection context
0928 16:41:51.678147 (+ 335us) client_negotiation.cc:240] Negotiation successful
0928 16:41:51.678812 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":319,"threads_started":1}
I20260928 16:41:51.682163 28806 heartbeater.cc:347] Connected to a master server at 127.10.247.254:41741
I20260928 16:41:51.682538 28806 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:51.683532 28806 heartbeater.cc:511] Master 127.10.247.254:41741 requested a full tablet report, sending...
I20260928 16:41:51.685467 28483 ts_manager.cc:194] Registered new tserver with Master: 7a4f7dde68dd4456bf6fab497d69ee05 (127.10.247.194:34315)
I20260928 16:41:51.686658 28483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:58605
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:41:51.737110 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:41741
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:41:52.066074 28672 heartbeater.cc:503] Master 127.10.247.254:41741 was elected leader, sending a full tablet report...
W20260928 16:41:52.146458 28812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:52.146890 28812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:52.147025 28812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:52.155179 28812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:52.155330 28812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:52.155469 28812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:41:52.166738 28812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:41741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:52.168663 28812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:52.170433 28812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:52.187961 28818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:52.188045 28817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:52.188094 28820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:52.188445 28812 server_base.cc:1034] running on GCE node
I20260928 16:41:52.189551 28812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:52.190961 28812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:52.192240 28812 hybrid_clock.cc:648] HybridClock initialized: now 1790613712192153 us; error 74 us; skew 500 ppm
I20260928 16:41:52.192721 28812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:52 dist-test-slave-9fn6 krb5kdc[28438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613712, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:52.196797 28812 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:41:52.198836 28812 webserver.cc:460] Webserver started at http://127.10.247.195:38279/ using document root <none> and password file <none>
I20260928 16:41:52.199812 28812 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:52.199983 28812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:52.200371 28812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:52.204330 28812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "992c889355b1400a9d6768df40980e31"
format_stamp: "Formatted at 2026-09-28 16:41:52 on dist-test-slave-9fn6"
server_key: "ad2b03b8ba4a59aeef33ea0d46cb30a9"
server_key_iv: "771853a9fce4c87a32a8caf36ae1ef15"
server_key_version: "encryptionkey@0"
I20260928 16:41:52.205396 28812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "992c889355b1400a9d6768df40980e31"
format_stamp: "Formatted at 2026-09-28 16:41:52 on dist-test-slave-9fn6"
server_key: "ad2b03b8ba4a59aeef33ea0d46cb30a9"
server_key_iv: "771853a9fce4c87a32a8caf36ae1ef15"
server_key_version: "encryptionkey@0"
I20260928 16:41:52.212189 28812 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:41:52.216823 28827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:52.218832 28812 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260928 16:41:52.219056 28812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "992c889355b1400a9d6768df40980e31"
format_stamp: "Formatted at 2026-09-28 16:41:52 on dist-test-slave-9fn6"
server_key: "ad2b03b8ba4a59aeef33ea0d46cb30a9"
server_key_iv: "771853a9fce4c87a32a8caf36ae1ef15"
server_key_version: "encryptionkey@0"
I20260928 16:41:52.219323 28812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:52.232757 28812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:52.237902 28812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:52.238236 28812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:52.239452 28812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:52.241278 28812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:52.241381 28812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:52.241533 28812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:52.241595 28812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:52.276263 28812 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:36519
I20260928 16:41:52.276269 28939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:36519 every 8 connection(s)
I20260928 16:41:52.278124 28812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:41:52.282186 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 28812
I20260928 16:41:52.282486 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:41:52.283242 11231 external_mini_cluster.cc:1428] Setting key 8701299290607384c519c0276ce11a83
Sep 28 16:41:52 dist-test-slave-9fn6 krb5kdc[28438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613712, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:41:52.305414 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.281853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:43793 (local address 127.10.247.254:41741)
0928 16:41:52.285652 (+ 3799us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.285659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.286239 (+ 580us) server_negotiation.cc:407] Connection header received
0928 16:41:52.288670 (+ 2431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.288676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.288771 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.288876 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:52.290771 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.291723 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.293499 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.294000 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.294122 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.298454 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:52.298461 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:52.298465 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:52.298510 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:52.301603 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.302601 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.302607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.302609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.302779 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.303276 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.303282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.303284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.303648 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:52.303803 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.304762 (+ 959us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.305062 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3619,"thread_start_us":3447,"threads_started":1}
I20260928 16:41:52.306591 28943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.282535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:41741 (local address 127.10.247.195:43793)
0928 16:41:52.286035 (+ 3500us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:52.286106 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:52.288285 (+ 2179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:52.289127 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:52.289138 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:52.289856 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:52.290587 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.290607 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.291882 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.291886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.293345 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.293358 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.294271 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.294279 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.294536 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.295427 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:41:52.295456 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:52.298142 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:52.301781 (+ 3639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:52.301792 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:52.301813 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:52.302421 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:52.302877 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:52.302880 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:52.302883 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:52.303119 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:52.303768 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:52.303776 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:52.304259 (+ 483us) client_negotiation.cc:769] Sending connection context
0928 16:41:52.304545 (+ 286us) client_negotiation.cc:240] Negotiation successful
0928 16:41:52.305157 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3124,"thread_start_us":2161,"threads_started":1}
I20260928 16:41:52.308369 28940 heartbeater.cc:347] Connected to a master server at 127.10.247.254:41741
I20260928 16:41:52.308837 28940 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:52.309772 28940 heartbeater.cc:511] Master 127.10.247.254:41741 requested a full tablet report, sending...
I20260928 16:41:52.311726 28483 ts_manager.cc:194] Registered new tserver with Master: 992c889355b1400a9d6768df40980e31 (127.10.247.195:36519)
I20260928 16:41:52.312916 28483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:43793
I20260928 16:41:52.313411 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:41:52.334111 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.319594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36456 (local address 127.10.247.254:41741)
0928 16:41:52.319874 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.319881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.320191 (+ 310us) server_negotiation.cc:407] Connection header received
0928 16:41:52.320402 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.320407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.320486 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.320575 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:52.322304 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.323149 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.324699 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.325210 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.325333 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.328050 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:52.328056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:52.328060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:52.328102 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:52.330778 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.331680 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.331684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.331687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.331848 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.332303 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.332308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.332311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.332824 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:52.333050 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.333501 (+ 451us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.333748 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260928 16:41:52.339610 28483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:41:52.348084 28483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:41:52.378032 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.364340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44451 (local address 127.0.0.1:39910)
0928 16:41:52.366583 (+ 2243us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:52.366642 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:52.366866 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:52.368122 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:52.368130 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:52.368166 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:52.369994 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.370007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.372209 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.372215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.374396 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.374410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.376734 (+ 2324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.376741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.376977 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.377061 (+ 84us) client_negotiation.cc:769] Sending connection context
0928 16:41:52.377211 (+ 150us) client_negotiation.cc:240] Negotiation successful
0928 16:41:52.377483 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1929,"mutex_wait_us":65,"thread_start_us":4440,"threads_started":1}
I20260928 16:41:52.378489 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.368080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34315 (local address 127.0.0.1:53240)
0928 16:41:52.369960 (+ 1880us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:52.369981 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:52.370118 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:52.370912 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:52.370917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:52.370941 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:52.371278 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.371289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.373164 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.373174 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.375418 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.375428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.377561 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.377566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.377703 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.377733 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 16:41:52.377816 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 16:41:52.377982 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1730,"mutex_wait_us":27,"thread_start_us":1577,"threads_started":1}
I20260928 16:41:52.378659 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.364562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52582 (local address 127.10.247.195:36519)
0928 16:41:52.366173 (+ 1611us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.366186 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.369322 (+ 3136us) server_negotiation.cc:407] Connection header received
0928 16:41:52.369692 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.369703 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.370032 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.370245 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:52.370897 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.372459 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.375276 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.376849 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.377056 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.377241 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.377848 (+ 607us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.378056 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1352,"thread_start_us":1089,"threads_started":1}
I20260928 16:41:52.378765 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.365203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.10.247.193:44451)
0928 16:41:52.366784 (+ 1581us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.366794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.366834 (+ 40us) server_negotiation.cc:407] Connection header received
0928 16:41:52.367198 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.367207 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.367548 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.367761 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:52.370240 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.371993 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.374589 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.376032 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.376275 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.376511 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.378187 (+ 1676us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.378401 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1387,"thread_start_us":1178,"threads_started":1}
I20260928 16:41:52.379267 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.367560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.10.247.194:34315)
0928 16:41:52.369446 (+ 1886us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.369456 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.370109 (+ 653us) server_negotiation.cc:407] Connection header received
0928 16:41:52.370347 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.370355 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.370591 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.370722 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:52.371460 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.373012 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.375571 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.377086 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.377259 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.377414 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.378791 (+ 1377us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.378994 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1723,"thread_start_us":1574,"threads_started":1}
I20260928 16:41:52.379626 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.364444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:36519 (local address 127.0.0.1:52582)
0928 16:41:52.369153 (+ 4709us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:52.369174 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:52.369449 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:52.370222 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:52.370226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:52.370254 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:52.370674 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.370687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.372661 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.372666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.375088 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.375108 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.377393 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.377399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.377610 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.377649 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 16:41:52.377748 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 16:41:52.377902 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4551,"mutex_wait_us":111,"thread_start_us":1677,"threads_started":1}
I20260928 16:41:52.383713 28607 tablet_service.cc:1467] Processing CreateTablet for tablet 8244f4dc9981462a895a8998ad1c347b (DEFAULT_TABLE table=test-table [id=840964d32fb04aeea532d8f0ad31ca3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:41:52.383724 28875 tablet_service.cc:1467] Processing CreateTablet for tablet 8244f4dc9981462a895a8998ad1c347b (DEFAULT_TABLE table=test-table [id=840964d32fb04aeea532d8f0ad31ca3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:41:52.383724 28741 tablet_service.cc:1467] Processing CreateTablet for tablet 8244f4dc9981462a895a8998ad1c347b (DEFAULT_TABLE table=test-table [id=840964d32fb04aeea532d8f0ad31ca3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:41:52.385468 28741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8244f4dc9981462a895a8998ad1c347b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:52.385458 28607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8244f4dc9981462a895a8998ad1c347b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:52.385464 28875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8244f4dc9981462a895a8998ad1c347b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:52.400074 28959 tablet_bootstrap.cc:492] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: Bootstrap starting.
I20260928 16:41:52.404069 28959 tablet_bootstrap.cc:654] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:52.404594 28961 tablet_bootstrap.cc:492] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: Bootstrap starting.
I20260928 16:41:52.406183 28960 tablet_bootstrap.cc:492] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: Bootstrap starting.
I20260928 16:41:52.407454 28959 log.cc:824] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:52.409217 28961 tablet_bootstrap.cc:654] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:52.409679 28960 tablet_bootstrap.cc:654] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:52.410430 28961 log.cc:824] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:52.410926 28960 log.cc:824] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:52.411569 28959 tablet_bootstrap.cc:492] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: No bootstrap required, opened a new log
I20260928 16:41:52.411893 28959 ts_tablet_manager.cc:1397] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260928 16:41:52.413812 28961 tablet_bootstrap.cc:492] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: No bootstrap required, opened a new log
I20260928 16:41:52.414134 28961 ts_tablet_manager.cc:1397] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260928 16:41:52.414904 28960 tablet_bootstrap.cc:492] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: No bootstrap required, opened a new log
I20260928 16:41:52.415194 28960 ts_tablet_manager.cc:1397] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 16:41:52.421324 28959 raft_consensus.cc:348] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } }
I20260928 16:41:52.421832 28959 raft_consensus.cc:374] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:52.421947 28959 raft_consensus.cc:729] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06c76dd10a544f05a3c84f3ff7c47e00, State: Initialized, Role: FOLLOWER
I20260928 16:41:52.422680 28959 consensus_queue.cc:260] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } }
I20260928 16:41:52.425712 28961 raft_consensus.cc:348] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } }
I20260928 16:41:52.426298 28961 raft_consensus.cc:374] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:52.426412 28961 raft_consensus.cc:729] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992c889355b1400a9d6768df40980e31, State: Initialized, Role: FOLLOWER
I20260928 16:41:52.426262 28960 raft_consensus.cc:348] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } }
I20260928 16:41:52.426728 28960 raft_consensus.cc:374] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:52.426826 28960 raft_consensus.cc:729] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4f7dde68dd4456bf6fab497d69ee05, State: Initialized, Role: FOLLOWER
I20260928 16:41:52.427179 28961 consensus_queue.cc:260] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } }
I20260928 16:41:52.427417 28960 consensus_queue.cc:260] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } }
I20260928 16:41:52.428786 28806 heartbeater.cc:503] Master 127.10.247.254:41741 was elected leader, sending a full tablet report...
I20260928 16:41:52.429364 28959 ts_tablet_manager.cc:1428] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260928 16:41:52.432986 28960 ts_tablet_manager.cc:1428] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: Time spent starting tablet: real 0.018s user 0.011s sys 0.006s
I20260928 16:41:52.433624 28940 heartbeater.cc:503] Master 127.10.247.254:41741 was elected leader, sending a full tablet report...
I20260928 16:41:52.434170 28961 ts_tablet_manager.cc:1428] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: Time spent starting tablet: real 0.020s user 0.013s sys 0.007s
I20260928 16:41:52.438701 28967 raft_consensus.cc:482] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:41:52.439016 28967 raft_consensus.cc:504] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } }
I20260928 16:41:52.441119 28967 leader_election.cc:288] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 992c889355b1400a9d6768df40980e31 (127.10.247.195:36519), 06c76dd10a544f05a3c84f3ff7c47e00 (127.10.247.193:44451)
I20260928 16:41:52.449169 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.441494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:38199 (local address 127.10.247.195:36519)
0928 16:41:52.441744 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.441752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.442745 (+ 993us) server_negotiation.cc:407] Connection header received
0928 16:41:52.442982 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.442987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.443058 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.443142 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:52.443899 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.444819 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.446962 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.447979 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.448124 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.448192 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.448703 (+ 511us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.448858 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:41:52.449321 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.441579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:51503 (local address 127.10.247.193:44451)
0928 16:41:52.441985 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.441993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.442795 (+ 802us) server_negotiation.cc:407] Connection header received
0928 16:41:52.443028 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.443033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.443103 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.443179 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:52.443942 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.444751 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.446929 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.448274 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.448462 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.448550 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.448852 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.449026 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260928 16:41:52.449347 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.441599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:36519 (local address 127.10.247.194:38199)
0928 16:41:52.442616 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:52.442641 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:52.442817 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:52.443169 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:52.443175 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:52.443201 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:52.443733 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.443746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.444950 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.444956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.446811 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.446828 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.448234 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.448241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.448442 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.448502 (+ 60us) client_negotiation.cc:769] Sending connection context
0928 16:41:52.448635 (+ 133us) client_negotiation.cc:240] Negotiation successful
0928 16:41:52.448776 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":21,"thread_start_us":730,"threads_started":1}
I20260928 16:41:52.449383 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.441623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:44451 (local address 127.10.247.194:51503)
0928 16:41:52.442682 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:52.442698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:52.442880 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:52.443205 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:52.443210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:52.443231 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:52.443792 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.443803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.444882 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.444886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.446722 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:52.446743 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.448486 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.448491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:52.448691 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.448724 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:41:52.448808 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 16:41:52.448932 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":44,"spinlock_wait_cycles":28928,"thread_start_us":446,"threads_started":1}
I20260928 16:41:52.450598 28627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8244f4dc9981462a895a8998ad1c347b" candidate_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" is_pre_election: true
I20260928 16:41:52.450603 28895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8244f4dc9981462a895a8998ad1c347b" candidate_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992c889355b1400a9d6768df40980e31" is_pre_election: true
I20260928 16:41:52.451226 28627 raft_consensus.cc:2454] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a4f7dde68dd4456bf6fab497d69ee05 in term 0.
I20260928 16:41:52.451288 28895 raft_consensus.cc:2454] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a4f7dde68dd4456bf6fab497d69ee05 in term 0.
I20260928 16:41:52.451992 28694 leader_election.cc:302] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06c76dd10a544f05a3c84f3ff7c47e00, 7a4f7dde68dd4456bf6fab497d69ee05; no voters:
I20260928 16:41:52.452536 28967 raft_consensus.cc:2781] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:41:52.452755 28967 raft_consensus.cc:482] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:41:52.452875 28967 raft_consensus.cc:3037] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:52.455681 28967 raft_consensus.cc:504] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } }
I20260928 16:41:52.456425 28967 leader_election.cc:288] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [CANDIDATE]: Term 1 election: Requested vote from peers 992c889355b1400a9d6768df40980e31 (127.10.247.195:36519), 06c76dd10a544f05a3c84f3ff7c47e00 (127.10.247.193:44451)
I20260928 16:41:52.457103 28895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8244f4dc9981462a895a8998ad1c347b" candidate_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992c889355b1400a9d6768df40980e31"
I20260928 16:41:52.457116 28627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8244f4dc9981462a895a8998ad1c347b" candidate_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06c76dd10a544f05a3c84f3ff7c47e00"
I20260928 16:41:52.457337 28895 raft_consensus.cc:3037] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:52.457368 28627 raft_consensus.cc:3037] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:52.459931 28627 raft_consensus.cc:2454] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a4f7dde68dd4456bf6fab497d69ee05 in term 1.
I20260928 16:41:52.459933 28895 raft_consensus.cc:2454] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a4f7dde68dd4456bf6fab497d69ee05 in term 1.
I20260928 16:41:52.460465 28694 leader_election.cc:302] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06c76dd10a544f05a3c84f3ff7c47e00, 7a4f7dde68dd4456bf6fab497d69ee05; no voters:
I20260928 16:41:52.460857 28967 raft_consensus.cc:2781] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:41:52.461450 28967 raft_consensus.cc:686] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 1 LEADER]: Becoming Leader. State: Replica: 7a4f7dde68dd4456bf6fab497d69ee05, State: Running, Role: LEADER
I20260928 16:41:52.462121 28967 consensus_queue.cc:237] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } }
I20260928 16:41:52.468077 28481 catalog_manager.cc:5495] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a4f7dde68dd4456bf6fab497d69ee05 (127.10.247.194). New cstate: current_term: 1 leader_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a4f7dde68dd4456bf6fab497d69ee05" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34315 } health_report { overall_health: HEALTHY } } }
I20260928 16:41:52.507360 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.501540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.10.247.194:34315)
0928 16:41:52.501758 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.501763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.501809 (+ 46us) server_negotiation.cc:407] Connection header received
0928 16:41:52.502071 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.502076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.502187 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.502338 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:41:52.502716 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.503514 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.505021 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.505484 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.505582 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.506098 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:41:52.506784 (+ 686us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:41:52.506914 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.507003 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.507091 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:41:52.517668 28895 raft_consensus.cc:1261] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [term 1 FOLLOWER]: Refusing update from remote peer 7a4f7dde68dd4456bf6fab497d69ee05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:41:52.517668 28627 raft_consensus.cc:1261] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [term 1 FOLLOWER]: Refusing update from remote peer 7a4f7dde68dd4456bf6fab497d69ee05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:41:52.518798 28970 consensus_queue.cc:1035] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992c889355b1400a9d6768df40980e31" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 36519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:41:52.519196 28967 consensus_queue.cc:1035] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c76dd10a544f05a3c84f3ff7c47e00" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 44451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 16:41:52.528398 28673 tablet.cc:2369] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:41:52.530210 28941 tablet.cc:2369] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:41:52.536902 28975 mvcc.cc:205] Tried to move back new op lower bound from 7334353766459891712 to 7334353766249406464. Current Snapshot: MvccSnapshot[applied={T|T < 7334353766459891712}]
I20260928 16:41:52.551498 28482 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:41:52.551865 28482 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:41:52.556694 28482 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 949664e70cca4b359055b9c7250cf59d: Sending DeleteTablet for 3 replicas of tablet 8244f4dc9981462a895a8998ad1c347b
I20260928 16:41:52.558290 28607 tablet_service.cc:1514] Processing DeleteTablet for tablet 8244f4dc9981462a895a8998ad1c347b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:41:52 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:39910
I20260928 16:41:52.558290 28875 tablet_service.cc:1514] Processing DeleteTablet for tablet 8244f4dc9981462a895a8998ad1c347b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:41:52 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:52582
I20260928 16:41:52.559258 28741 tablet_service.cc:1514] Processing DeleteTablet for tablet 8244f4dc9981462a895a8998ad1c347b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:41:52 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:53240
I20260928 16:41:52.559888 28982 tablet_replica.cc:330] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: stopping tablet replica
I20260928 16:41:52.560523 28982 raft_consensus.cc:2229] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:41:52.561312 28982 raft_consensus.cc:2258] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:41:52.562282 28983 tablet_replica.cc:330] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: stopping tablet replica
I20260928 16:41:52.562865 28983 raft_consensus.cc:2229] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:41:52.563556 28983 raft_consensus.cc:2258] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:41:52.563706 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 28544
I20260928 16:41:52.565943 28982 ts_tablet_manager.cc:1905] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:41:52.567691 28983 ts_tablet_manager.cc:1905] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:41:52.574282 28982 ts_tablet_manager.cc:1918] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:52.574615 28982 log.cc:1196] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/8244f4dc9981462a895a8998ad1c347b
I20260928 16:41:52.575336 28982 ts_tablet_manager.cc:1939] T 8244f4dc9981462a895a8998ad1c347b P 992c889355b1400a9d6768df40980e31: Deleting consensus metadata
I20260928 16:41:52.578058 28468 catalog_manager.cc:4841] TS 992c889355b1400a9d6768df40980e31 (127.10.247.195:36519): tablet 8244f4dc9981462a895a8998ad1c347b (table test-table [id=840964d32fb04aeea532d8f0ad31ca3e]) successfully deleted
I20260928 16:41:52.579133 28984 tablet_replica.cc:330] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: stopping tablet replica
I20260928 16:41:52.579764 28984 raft_consensus.cc:2229] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:41:52.580762 28984 raft_consensus.cc:2258] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:41:52.585076 28983 ts_tablet_manager.cc:1918] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:52.585310 28983 log.cc:1196] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/8244f4dc9981462a895a8998ad1c347b
I20260928 16:41:52.585935 28983 ts_tablet_manager.cc:1939] T 8244f4dc9981462a895a8998ad1c347b P 06c76dd10a544f05a3c84f3ff7c47e00: Deleting consensus metadata
I20260928 16:41:52.587327 28984 ts_tablet_manager.cc:1905] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 16:41:52 dist-test-slave-9fn6 krb5kdc[28438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:41:52.588464 28467 catalog_manager.cc:4841] TS 06c76dd10a544f05a3c84f3ff7c47e00 (127.10.247.193:44451): tablet 8244f4dc9981462a895a8998ad1c347b (table test-table [id=840964d32fb04aeea532d8f0ad31ca3e]) successfully deleted
I20260928 16:41:52.595075 28984 ts_tablet_manager.cc:1918] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:52.595335 28984 log.cc:1196] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/8244f4dc9981462a895a8998ad1c347b
I20260928 16:41:52.595923 28984 ts_tablet_manager.cc:1939] T 8244f4dc9981462a895a8998ad1c347b P 7a4f7dde68dd4456bf6fab497d69ee05: Deleting consensus metadata
I20260928 16:41:52.598248 28468 catalog_manager.cc:4841] TS 7a4f7dde68dd4456bf6fab497d69ee05 (127.10.247.194:34315): tablet 8244f4dc9981462a895a8998ad1c347b (table test-table [id=840964d32fb04aeea532d8f0ad31ca3e]) successfully deleted
I20260928 16:41:52.598600 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.565251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39924 (local address 127.10.247.193:44451)
0928 16:41:52.565457 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.565464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.576119 (+ 10655us) server_negotiation.cc:407] Connection header received
0928 16:41:52.576841 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.576850 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.576963 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.577048 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:52.579245 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.580203 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.582149 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.582701 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.582868 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.589778 (+ 6910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:52.589799 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:52.589824 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:52.589883 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:52.594452 (+ 4569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.595613 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.595626 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.595630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.595829 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.596386 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.596392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.596395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.597037 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:52.597395 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.597797 (+ 402us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.598089 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 16:41:52.599478 28667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:52.700639 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 28544
I20260928 16:41:52.719628 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 28678
Sep 28 16:41:52 dist-test-slave-9fn6 krb5kdc[28438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:41:52.738710 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.720604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53258 (local address 127.10.247.194:34315)
0928 16:41:52.721201 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.721206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.721224 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:41:52.721296 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.721300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.721376 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.721471 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:52.723046 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.724021 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.725926 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.726458 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.726599 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.731825 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:52.731867 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:52.731890 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:52.731958 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:52.735425 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.736236 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.736245 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.736247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.736433 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.736896 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.736900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.736902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.737355 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:52.737752 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.738018 (+ 266us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.738275 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420}
I20260928 16:41:52.739550 28801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:52.870577 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 28678
I20260928 16:41:52.889222 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 28812
Sep 28 16:41:52 dist-test-slave-9fn6 krb5kdc[28438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:41:52.907646 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:52.890099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52588 (local address 127.10.247.195:36519)
0928 16:41:52.890400 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:52.890406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:52.890431 (+ 25us) server_negotiation.cc:407] Connection header received
0928 16:41:52.890509 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:52.890513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:52.890605 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:52.890705 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:52.892359 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.893259 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.894708 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:52.895171 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:52.895277 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:52.900515 (+ 5238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:52.900538 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:52.900562 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:52.900759 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:52.903933 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.904963 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.904975 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.904978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.905229 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:52.905753 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:52.905758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:52.905760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:52.906298 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:52.906586 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:52.906991 (+ 405us) server_negotiation.cc:299] Negotiation successful
0928 16:41:52.907242 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260928 16:41:52.908432 28935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:53.006223 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 28812
I20260928 16:41:53.023396 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 28451
I20260928 16:41:53.024955 28513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:53.154290 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 28451
2026-09-28T16:41:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4817 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-28T16:41:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:41:53Z Disabled control of system clock
I20260928 16:41:53.246562 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:36215
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:36215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:41:53.649014 28993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:53.649415 28993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:53.649507 28993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:53.657658 28993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:41:53.657820 28993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:53.657859 28993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:53.657922 28993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:41:53.657968 28993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:41:53.668515 28993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:36215
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:36215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:53.670349 28993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:53.672585 28993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:53.686394 28998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:53.686338 29001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:53.686368 28999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:53.687418 28993 server_base.cc:1034] running on GCE node
I20260928 16:41:53.688364 28993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:53.690239 28993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:53.691464 28993 hybrid_clock.cc:648] HybridClock initialized: now 1790613713691412 us; error 47 us; skew 500 ppm
I20260928 16:41:53.691871 28993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:41:53.699007 28993 webserver.cc:460] Webserver started at http://127.10.247.254:39625/ using document root <none> and password file <none>
I20260928 16:41:53.700012 28993 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:53.700141 28993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:53.700526 28993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:53.704588 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "2a5bea871bd94881a7c5605a40229a39"
format_stamp: "Formatted at 2026-09-28 16:41:53 on dist-test-slave-9fn6"
server_key: "c13e96e68f5c7d76a0dcf222602c257d"
server_key_iv: "4451e572b6862d105164151ca3ba9c48"
server_key_version: "encryptionkey@0"
I20260928 16:41:53.705663 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "2a5bea871bd94881a7c5605a40229a39"
format_stamp: "Formatted at 2026-09-28 16:41:53 on dist-test-slave-9fn6"
server_key: "c13e96e68f5c7d76a0dcf222602c257d"
server_key_iv: "4451e572b6862d105164151ca3ba9c48"
server_key_version: "encryptionkey@0"
I20260928 16:41:53.712096 28993 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260928 16:41:53.716440 29007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:53.718182 28993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:41:53.718389 28993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "2a5bea871bd94881a7c5605a40229a39"
format_stamp: "Formatted at 2026-09-28 16:41:53 on dist-test-slave-9fn6"
server_key: "c13e96e68f5c7d76a0dcf222602c257d"
server_key_iv: "4451e572b6862d105164151ca3ba9c48"
server_key_version: "encryptionkey@0"
I20260928 16:41:53.718643 28993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:53.737358 28993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:53.738483 28993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:53.738822 28993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:53.759167 28993 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:36215
I20260928 16:41:53.759184 29058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:36215 every 8 connection(s)
I20260928 16:41:53.760854 28993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:41:53.761464 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 28993
I20260928 16:41:53.761647 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:41:53.762352 11231 external_mini_cluster.cc:1428] Setting key eb14bccca576575c8af6d8084a060f57
I20260928 16:41:53.765898 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:53.778087 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:53.764652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.10.247.254:36215)
0928 16:41:53.765754 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:53.765768 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:53.765817 (+ 49us) server_negotiation.cc:407] Connection header received
0928 16:41:53.767651 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:53.767675 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:53.768204 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:53.768574 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:53.769322 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:53.770850 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:53.773121 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:53.773856 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:53.774016 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:53.775319 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:53.775327 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:53.775343 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:41:53.775981 (+ 638us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:53.776085 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:41:53.776109 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:53.776209 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:53.776452 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 16:41:53.776881 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":389,"threads_started":1}
I20260928 16:41:53.779840 29059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39: Bootstrap starting.
I20260928 16:41:53.783723 29059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:53.784904 29059 log.cc:824] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:53.788743 29059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39: No bootstrap required, opened a new log
I20260928 16:41:53.798004 29059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5bea871bd94881a7c5605a40229a39" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36215 } }
I20260928 16:41:53.798681 29059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:53.798830 29059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5bea871bd94881a7c5605a40229a39, State: Initialized, Role: FOLLOWER
I20260928 16:41:53.799676 29059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5bea871bd94881a7c5605a40229a39" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36215 } }
I20260928 16:41:53.799942 29059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:41:53.800099 29059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:41:53.800308 29059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:53.803064 29059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5bea871bd94881a7c5605a40229a39" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36215 } }
I20260928 16:41:53.803853 29059 leader_election.cc:302] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a5bea871bd94881a7c5605a40229a39; no voters:
I20260928 16:41:53.804689 29059 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:41:53.804937 29064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:41:53.805866 29064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [term 1 LEADER]: Becoming Leader. State: Replica: 2a5bea871bd94881a7c5605a40229a39, State: Running, Role: LEADER
I20260928 16:41:53.806591 29064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5bea871bd94881a7c5605a40229a39" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36215 } }
I20260928 16:41:53.807288 29059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:41:53.810557 29065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a5bea871bd94881a7c5605a40229a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5bea871bd94881a7c5605a40229a39" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36215 } } }
I20260928 16:41:53.810505 29066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a5bea871bd94881a7c5605a40229a39. Latest consensus state: current_term: 1 leader_uuid: "2a5bea871bd94881a7c5605a40229a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5bea871bd94881a7c5605a40229a39" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 36215 } } }
I20260928 16:41:53.811188 29065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:53.811247 29066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39 [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:53.812400 29073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:41:53.826282 29073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:41:53.840310 29073 catalog_manager.cc:1347] Generated new cluster ID: 8f10ced6890c4802b40707d38406a96d
I20260928 16:41:53.840464 29073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:41:53.870504 29073 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:41:53.872161 29073 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:41:53.883625 29073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39: Generated new TSK 0
I20260928 16:41:53.884900 29073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:41:53.895731 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36215
--builtin_ntp_servers=127.10.247.212:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:41:54.332217 29083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:54.332577 29083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:54.332726 29083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:54.340751 29083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:54.340888 29083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:54.341051 29083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:41:54.351311 29083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:54.353121 29083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:54.355500 29083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:54.370942 29088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:54.371076 29091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:54.370961 29089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:54.374563 29083 server_base.cc:1034] running on GCE node
I20260928 16:41:54.375383 29083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:54.376550 29083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:54.377823 29083 hybrid_clock.cc:648] HybridClock initialized: now 1790613714377762 us; error 44 us; skew 500 ppm
I20260928 16:41:54.378203 29083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:41:54.380898 29083 webserver.cc:460] Webserver started at http://127.10.247.193:38693/ using document root <none> and password file <none>
I20260928 16:41:54.381757 29083 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:54.381873 29083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:54.382254 29083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:54.385946 29083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "afe51739221e489ab5e6d60607fa11c3"
format_stamp: "Formatted at 2026-09-28 16:41:54 on dist-test-slave-9fn6"
server_key: "8ba4fb638217abb75b7c5671eb370ab6"
server_key_iv: "dc1af0b596590395d7660c282a77de86"
server_key_version: "encryptionkey@0"
I20260928 16:41:54.386898 29083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "afe51739221e489ab5e6d60607fa11c3"
format_stamp: "Formatted at 2026-09-28 16:41:54 on dist-test-slave-9fn6"
server_key: "8ba4fb638217abb75b7c5671eb370ab6"
server_key_iv: "dc1af0b596590395d7660c282a77de86"
server_key_version: "encryptionkey@0"
I20260928 16:41:54.394196 29083 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260928 16:41:54.398872 29097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:54.400835 29083 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260928 16:41:54.401041 29083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "afe51739221e489ab5e6d60607fa11c3"
format_stamp: "Formatted at 2026-09-28 16:41:54 on dist-test-slave-9fn6"
server_key: "8ba4fb638217abb75b7c5671eb370ab6"
server_key_iv: "dc1af0b596590395d7660c282a77de86"
server_key_version: "encryptionkey@0"
I20260928 16:41:54.401309 29083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:54.428508 29083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:54.429759 29083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:54.430130 29083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:54.431383 29083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:54.433245 29083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:54.433341 29083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:54.433454 29083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:54.433532 29083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:54.467388 29083 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:37723
I20260928 16:41:54.467451 29209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:37723 every 8 connection(s)
I20260928 16:41:54.469045 29083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:41:54.471200 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 29083
I20260928 16:41:54.471411 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:41:54.471952 11231 external_mini_cluster.cc:1428] Setting key a18ed149a83d819d71567c5bc11d209c
I20260928 16:41:54.475442 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36215
--builtin_ntp_servers=127.10.247.212:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:41:54.485268 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:54.472507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:60279 (local address 127.10.247.254:36215)
0928 16:41:54.473099 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:54.473105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:54.474881 (+ 1776us) server_negotiation.cc:407] Connection header received
0928 16:41:54.477316 (+ 2435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:54.477322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:54.477437 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:54.477553 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:54.478966 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:54.479996 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:54.482042 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:54.482625 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:54.482748 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:54.484335 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:54.484341 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:54.484346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:41:54.484373 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:54.484427 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:41:54.484444 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:54.484528 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:54.484769 (+ 241us) server_negotiation.cc:299] Negotiation successful
0928 16:41:54.484888 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":315,"threads_started":1}
I20260928 16:41:54.486526 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:54.473354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:36215 (local address 127.10.247.193:60279)
0928 16:41:54.474676 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:54.474743 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:54.476957 (+ 2214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:54.477838 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:54.477848 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:54.477983 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:54.478767 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:54.478793 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:54.480181 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:54.480185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:54.481861 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:54.481876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:54.482872 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:54.482877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:54.483158 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:54.483940 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:41:54.483944 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:54.483995 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:41:54.484128 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:54.484533 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:54.484540 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:54.484546 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 16:41:54.484749 (+ 203us) client_negotiation.cc:240] Negotiation successful
0928 16:41:54.485179 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":393,"threads_started":1}
I20260928 16:41:54.488373 29210 heartbeater.cc:347] Connected to a master server at 127.10.247.254:36215
I20260928 16:41:54.488790 29210 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:54.489886 29210 heartbeater.cc:511] Master 127.10.247.254:36215 requested a full tablet report, sending...
I20260928 16:41:54.492769 29024 ts_manager.cc:194] Registered new tserver with Master: afe51739221e489ab5e6d60607fa11c3 (127.10.247.193:37723)
I20260928 16:41:54.495123 29024 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.193:60279
W20260928 16:41:54.870536 29214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:54.870939 29214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:54.871045 29214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:54.879040 29214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:54.879189 29214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:54.879313 29214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:41:54.889487 29214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:54.891160 29214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:54.893684 29214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:54.909209 29220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:54.909547 29222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:54.909268 29219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:54.913630 29214 server_base.cc:1034] running on GCE node
I20260928 16:41:54.914449 29214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:54.915639 29214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:54.916855 29214 hybrid_clock.cc:648] HybridClock initialized: now 1790613714916798 us; error 43 us; skew 500 ppm
I20260928 16:41:54.917217 29214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:41:54.919862 29214 webserver.cc:460] Webserver started at http://127.10.247.194:45041/ using document root <none> and password file <none>
I20260928 16:41:54.920781 29214 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:54.920889 29214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:54.921213 29214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:54.925050 29214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "58848017efdd433092411764a9c2157b"
format_stamp: "Formatted at 2026-09-28 16:41:54 on dist-test-slave-9fn6"
server_key: "4af4b10370bf903a75f67e72395cb8b2"
server_key_iv: "7c96b7455d77faa70642206e24287b67"
server_key_version: "encryptionkey@0"
I20260928 16:41:54.925943 29214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "58848017efdd433092411764a9c2157b"
format_stamp: "Formatted at 2026-09-28 16:41:54 on dist-test-slave-9fn6"
server_key: "4af4b10370bf903a75f67e72395cb8b2"
server_key_iv: "7c96b7455d77faa70642206e24287b67"
server_key_version: "encryptionkey@0"
I20260928 16:41:54.932312 29214 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:41:54.936873 29228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:54.938602 29214 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:41:54.938791 29214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "58848017efdd433092411764a9c2157b"
format_stamp: "Formatted at 2026-09-28 16:41:54 on dist-test-slave-9fn6"
server_key: "4af4b10370bf903a75f67e72395cb8b2"
server_key_iv: "7c96b7455d77faa70642206e24287b67"
server_key_version: "encryptionkey@0"
I20260928 16:41:54.939055 29214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:54.967778 29214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:54.968981 29214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:54.969327 29214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:54.970654 29214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:54.972579 29214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:54.972700 29214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260928 16:41:54.972826 29214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:54.972898 29214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:55.009183 29214 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:44591
I20260928 16:41:55.009294 29340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:44591 every 8 connection(s)
I20260928 16:41:55.010842 29214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:41:55.015303 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 29214
I20260928 16:41:55.015501 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:41:55.016036 11231 external_mini_cluster.cc:1428] Setting key 60de9b295a95ba105fdc545813769298
I20260928 16:41:55.019796 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36215
--builtin_ntp_servers=127.10.247.212:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:41:55.027287 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.013952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:35845 (local address 127.10.247.254:36215)
0928 16:41:55.014493 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:55.014499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:55.016085 (+ 1586us) server_negotiation.cc:407] Connection header received
0928 16:41:55.018391 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:55.018397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:55.018495 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:55.018602 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:55.020336 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.021293 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.023666 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.024217 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.024334 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.026295 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:55.026301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:55.026305 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:41:55.026333 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:55.026394 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:41:55.026409 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:55.026504 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:55.026803 (+ 299us) server_negotiation.cc:299] Negotiation successful
0928 16:41:55.026932 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":283,"threads_started":1}
I20260928 16:41:55.029079 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.014625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:36215 (local address 127.10.247.194:35845)
0928 16:41:55.015862 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:55.015916 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:55.018040 (+ 2124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:55.018916 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:55.018929 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:55.019082 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:55.020079 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.020112 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.021475 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.021480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.023479 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.023497 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.024484 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.024490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.024876 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.025806 (+ 930us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:41:55.025810 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:55.025877 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:41:55.026023 (+ 146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:55.026512 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:55.026523 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:55.026532 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 16:41:55.026775 (+ 243us) client_negotiation.cc:240] Negotiation successful
0928 16:41:55.027336 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":343,"threads_started":1}
I20260928 16:41:55.031103 29341 heartbeater.cc:347] Connected to a master server at 127.10.247.254:36215
I20260928 16:41:55.031531 29341 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:55.032824 29341 heartbeater.cc:511] Master 127.10.247.254:36215 requested a full tablet report, sending...
I20260928 16:41:55.034950 29024 ts_manager.cc:194] Registered new tserver with Master: 58848017efdd433092411764a9c2157b (127.10.247.194:44591)
I20260928 16:41:55.036002 29024 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.194:35845
W20260928 16:41:55.418543 29345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:55.418911 29345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:55.419034 29345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:55.427186 29345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:55.427328 29345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:55.427487 29345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:41:55.437903 29345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:36215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:55.439751 29345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:55.442121 29345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:55.457604 29350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:55.457851 29351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:55.457788 29353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:55.458994 29345 server_base.cc:1034] running on GCE node
I20260928 16:41:55.459753 29345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:55.461007 29345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:55.462255 29345 hybrid_clock.cc:648] HybridClock initialized: now 1790613715462181 us; error 50 us; skew 500 ppm
I20260928 16:41:55.462641 29345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:41:55.465368 29345 webserver.cc:460] Webserver started at http://127.10.247.195:42407/ using document root <none> and password file <none>
I20260928 16:41:55.466208 29345 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:55.466306 29345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:55.466714 29345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:55.471289 29345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "55b37ee5246744c7bed1f8579a81c222"
format_stamp: "Formatted at 2026-09-28 16:41:55 on dist-test-slave-9fn6"
server_key: "5a46b576ff3a7ea5a1c9d5d08d88ed4e"
server_key_iv: "018c3d543be9f0f97829892d10b911e2"
server_key_version: "encryptionkey@0"
I20260928 16:41:55.472553 29345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "55b37ee5246744c7bed1f8579a81c222"
format_stamp: "Formatted at 2026-09-28 16:41:55 on dist-test-slave-9fn6"
server_key: "5a46b576ff3a7ea5a1c9d5d08d88ed4e"
server_key_iv: "018c3d543be9f0f97829892d10b911e2"
server_key_version: "encryptionkey@0"
I20260928 16:41:55.479153 29345 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:41:55.483515 29359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:55.485301 29345 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:41:55.485493 29345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "55b37ee5246744c7bed1f8579a81c222"
format_stamp: "Formatted at 2026-09-28 16:41:55 on dist-test-slave-9fn6"
server_key: "5a46b576ff3a7ea5a1c9d5d08d88ed4e"
server_key_iv: "018c3d543be9f0f97829892d10b911e2"
server_key_version: "encryptionkey@0"
I20260928 16:41:55.485755 29345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:55.499861 29210 heartbeater.cc:503] Master 127.10.247.254:36215 was elected leader, sending a full tablet report...
I20260928 16:41:55.513098 29345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:55.514173 29345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:55.514509 29345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:55.515697 29345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:55.517562 29345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:55.517656 29345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:55.517774 29345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:55.517843 29345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:55.551966 29345 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:39859
I20260928 16:41:55.552016 29471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:39859 every 8 connection(s)
I20260928 16:41:55.553622 29345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:41:55.557786 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 29345
I20260928 16:41:55.557983 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:41:55.558521 11231 external_mini_cluster.cc:1428] Setting key 706c9f5cd510548f8be3fffaa7a2c764
I20260928 16:41:55.569166 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.556753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:43261 (local address 127.10.247.254:36215)
0928 16:41:55.557286 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:55.557293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:55.559120 (+ 1827us) server_negotiation.cc:407] Connection header received
0928 16:41:55.561597 (+ 2477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:55.561603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:55.561710 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:55.561841 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:55.563189 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.564030 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.565868 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.566340 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.566446 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.568165 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:55.568171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:55.568174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:41:55.568199 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:55.568258 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:41:55.568271 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:55.568349 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:55.568657 (+ 308us) server_negotiation.cc:299] Negotiation successful
0928 16:41:55.568779 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":236,"threads_started":1}
I20260928 16:41:55.570441 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.557527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:36215 (local address 127.10.247.195:43261)
0928 16:41:55.558885 (+ 1358us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:55.558941 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:55.561274 (+ 2333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:55.562146 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:55.562157 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:55.562278 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:55.563005 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.563024 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.564193 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.564197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.565716 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.565730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.566614 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.566620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.566872 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.567764 (+ 892us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:41:55.567767 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:55.567819 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:41:55.567934 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:55.568361 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:55.568370 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:55.568378 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 16:41:55.568585 (+ 207us) client_negotiation.cc:240] Negotiation successful
0928 16:41:55.569084 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":407,"threads_started":1}
I20260928 16:41:55.572220 29472 heartbeater.cc:347] Connected to a master server at 127.10.247.254:36215
I20260928 16:41:55.572639 29472 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:55.573729 29472 heartbeater.cc:511] Master 127.10.247.254:36215 requested a full tablet report, sending...
I20260928 16:41:55.575495 29024 ts_manager.cc:194] Registered new tserver with Master: 55b37ee5246744c7bed1f8579a81c222 (127.10.247.195:39859)
I20260928 16:41:55.576450 29024 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.195:43261
I20260928 16:41:55.587707 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:41:55.599294 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.592591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.10.247.254:36215)
0928 16:41:55.592843 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:55.592851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:55.593261 (+ 410us) server_negotiation.cc:407] Connection header received
0928 16:41:55.593462 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:55.593467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:55.593566 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:55.593672 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:55.594287 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.595100 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.596526 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.597051 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.597173 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.598369 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:55.598374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:55.598377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:41:55.598399 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:55.598438 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:41:55.598452 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:55.598529 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:55.598775 (+ 246us) server_negotiation.cc:299] Negotiation successful
0928 16:41:55.598887 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260928 16:41:55.603749 29024 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:41:55.611884 29024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:41:55.640969 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.626704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:39859 (local address 127.0.0.1:37144)
0928 16:41:55.627836 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:55.627891 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:55.628142 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:55.630391 (+ 2249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:55.630401 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:55.630440 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:55.630970 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.630985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.633907 (+ 2922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.633923 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.636768 (+ 2845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.636792 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.639729 (+ 2937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.639737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.640028 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.640105 (+ 77us) client_negotiation.cc:769] Sending connection context
0928 16:41:55.640288 (+ 183us) client_negotiation.cc:240] Negotiation successful
0928 16:41:55.640562 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"threads_started":1}
I20260928 16:41:55.642505 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.627428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.10.247.195:39859)
0928 16:41:55.629564 (+ 2136us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:55.629574 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:55.629614 (+ 40us) server_negotiation.cc:407] Connection header received
0928 16:41:55.629735 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:55.629744 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:55.630062 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:55.631221 (+ 1159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:55.631581 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.633700 (+ 2119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.637111 (+ 3411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.638881 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.641355 (+ 2474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.641700 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:55.641804 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 16:41:55.642042 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1253,"thread_start_us":1175,"threads_started":1}
I20260928 16:41:55.648784 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.629702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:44591 (local address 127.0.0.1:44408)
0928 16:41:55.641259 (+ 11557us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:55.641275 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:55.641370 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:55.642445 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:55.642450 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:55.642479 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:55.643066 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.643077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.644925 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.644930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.646474 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.646487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.647910 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.647916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.648101 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.648136 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 16:41:55.648213 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 16:41:55.648358 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11424,"mutex_wait_us":325,"thread_start_us":10579,"threads_started":1}
I20260928 16:41:55.648528 29407 tablet_service.cc:1467] Processing CreateTablet for tablet 82c39ee7f78741858df39d1dd971d6fc (DEFAULT_TABLE table=test-table [id=b5d1342488ea478c9310a19e153b468b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:41:55.650200 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.626953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37723 (local address 127.0.0.1:45662)
0928 16:41:55.640592 (+ 13639us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:55.640638 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:55.640914 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:55.642461 (+ 1547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:55.642465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:55.642507 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:55.643218 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.643228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.645394 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.645400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.647460 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:55.647476 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.649386 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.649392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:55.649574 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.649607 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:41:55.649703 (+ 96us) client_negotiation.cc:240] Negotiation successful
0928 16:41:55.649844 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13505,"mutex_wait_us":4,"spinlock_wait_cycles":3968,"thread_start_us":852,"threads_started":1}
I20260928 16:41:55.650421 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.630370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44408 (local address 127.10.247.194:44591)
0928 16:41:55.640108 (+ 9738us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:55.640117 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:55.641499 (+ 1382us) server_negotiation.cc:407] Connection header received
0928 16:41:55.641640 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:55.641646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:55.641906 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:55.642140 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:55.643228 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.644748 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.646648 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.647740 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.647918 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.648189 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:55.649817 (+ 1628us) server_negotiation.cc:299] Negotiation successful
0928 16:41:55.650036 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9549,"thread_start_us":9398,"threads_started":1}
I20260928 16:41:55.650468 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:55.627330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.10.247.193:37723)
0928 16:41:55.639599 (+ 12269us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:55.639611 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:55.640774 (+ 1163us) server_negotiation.cc:407] Connection header received
0928 16:41:55.641547 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:55.641559 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:55.641935 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:55.642140 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:55.643461 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.645218 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.647694 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:55.649180 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:55.649387 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:55.649620 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:55.649898 (+ 278us) server_negotiation.cc:299] Negotiation successful
0928 16:41:55.650080 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12036,"thread_start_us":11849,"threads_started":1}
I20260928 16:41:55.650693 29407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c39ee7f78741858df39d1dd971d6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:55.654528 29276 tablet_service.cc:1467] Processing CreateTablet for tablet 82c39ee7f78741858df39d1dd971d6fc (DEFAULT_TABLE table=test-table [id=b5d1342488ea478c9310a19e153b468b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:41:55.654526 29145 tablet_service.cc:1467] Processing CreateTablet for tablet 82c39ee7f78741858df39d1dd971d6fc (DEFAULT_TABLE table=test-table [id=b5d1342488ea478c9310a19e153b468b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:41:55.656010 29276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c39ee7f78741858df39d1dd971d6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:55.656138 29145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c39ee7f78741858df39d1dd971d6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:55.669989 29491 tablet_bootstrap.cc:492] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: Bootstrap starting.
I20260928 16:41:55.670449 29492 tablet_bootstrap.cc:492] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: Bootstrap starting.
I20260928 16:41:55.671540 29493 tablet_bootstrap.cc:492] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: Bootstrap starting.
I20260928 16:41:55.674465 29492 tablet_bootstrap.cc:654] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:55.674804 29491 tablet_bootstrap.cc:654] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:55.675721 29492 log.cc:824] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:55.675884 29493 tablet_bootstrap.cc:654] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:55.676079 29491 log.cc:824] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:55.677223 29493 log.cc:824] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: Log is configured to *not* fsync() on all Append() calls
I20260928 16:41:55.679973 29492 tablet_bootstrap.cc:492] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: No bootstrap required, opened a new log
I20260928 16:41:55.680156 29491 tablet_bootstrap.cc:492] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: No bootstrap required, opened a new log
I20260928 16:41:55.680292 29492 ts_tablet_manager.cc:1397] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260928 16:41:55.680451 29491 ts_tablet_manager.cc:1397] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260928 16:41:55.681082 29493 tablet_bootstrap.cc:492] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: No bootstrap required, opened a new log
I20260928 16:41:55.681370 29493 ts_tablet_manager.cc:1397] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260928 16:41:55.688443 29492 raft_consensus.cc:348] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } }
I20260928 16:41:55.689316 29492 raft_consensus.cc:374] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:55.689515 29492 raft_consensus.cc:729] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58848017efdd433092411764a9c2157b, State: Initialized, Role: FOLLOWER
I20260928 16:41:55.690631 29492 consensus_queue.cc:260] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } }
I20260928 16:41:55.692088 29341 heartbeater.cc:503] Master 127.10.247.254:36215 was elected leader, sending a full tablet report...
I20260928 16:41:55.692093 29491 raft_consensus.cc:348] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } }
I20260928 16:41:55.692094 29493 raft_consensus.cc:348] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } }
I20260928 16:41:55.692591 29491 raft_consensus.cc:374] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:55.692591 29493 raft_consensus.cc:374] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:55.692708 29491 raft_consensus.cc:729] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b37ee5246744c7bed1f8579a81c222, State: Initialized, Role: FOLLOWER
I20260928 16:41:55.692751 29493 raft_consensus.cc:729] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe51739221e489ab5e6d60607fa11c3, State: Initialized, Role: FOLLOWER
I20260928 16:41:55.693192 29492 ts_tablet_manager.cc:1428] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: Time spent starting tablet: real 0.013s user 0.005s sys 0.009s
I20260928 16:41:55.693569 29493 consensus_queue.cc:260] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } }
I20260928 16:41:55.693580 29491 consensus_queue.cc:260] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } }
I20260928 16:41:55.694813 29472 heartbeater.cc:503] Master 127.10.247.254:36215 was elected leader, sending a full tablet report...
I20260928 16:41:55.695596 29491 ts_tablet_manager.cc:1428] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260928 16:41:55.697387 29493 ts_tablet_manager.cc:1428] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260928 16:41:55.722004 29211 tablet.cc:2369] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:41:55.763440 29342 tablet.cc:2369] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:41:55.805540 29473 tablet.cc:2369] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:41:56.018237 29499 raft_consensus.cc:482] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:41:56.018672 29499 raft_consensus.cc:504] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } }
I20260928 16:41:56.020826 29499 leader_election.cc:288] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55b37ee5246744c7bed1f8579a81c222 (127.10.247.195:39859), 58848017efdd433092411764a9c2157b (127.10.247.194:44591)
I20260928 16:41:56.030761 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.021595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:47697 (local address 127.10.247.195:39859)
0928 16:41:56.021901 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:56.021908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:56.022410 (+ 502us) server_negotiation.cc:407] Connection header received
0928 16:41:56.022690 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:56.022697 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:56.022786 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:56.022873 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:56.023670 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.024906 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.028308 (+ 3402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.029627 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.029811 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:56.029896 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:56.030315 (+ 419us) server_negotiation.cc:299] Negotiation successful
0928 16:41:56.030460 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260928 16:41:56.030761 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.021555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:54239 (local address 127.10.247.194:44591)
0928 16:41:56.021878 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:56.021886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:56.022595 (+ 709us) server_negotiation.cc:407] Connection header received
0928 16:41:56.022786 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:56.022792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:56.022887 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:56.022973 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:41:56.023663 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.024898 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.028299 (+ 3401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.029626 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.029812 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:56.029896 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:56.030315 (+ 419us) server_negotiation.cc:299] Negotiation successful
0928 16:41:56.030461 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260928 16:41:56.030926 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.021362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:39859 (local address 127.10.247.193:47697)
0928 16:41:56.022279 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:56.022298 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:56.022487 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:56.022914 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:56.022918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:56.022943 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:56.023457 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:56.023475 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.025076 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.025083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:56.028027 (+ 2944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:56.028049 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.029883 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.029890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:56.030124 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:56.030180 (+ 56us) client_negotiation.cc:769] Sending connection context
0928 16:41:56.030279 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 16:41:56.030488 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":220,"thread_start_us":512,"threads_started":1}
I20260928 16:41:56.030956 29501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.021362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:44591 (local address 127.10.247.193:54239)
0928 16:41:56.022478 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:56.022493 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:56.022643 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:56.022993 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:56.022997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:56.023021 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:41:56.023508 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:56.023517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.025077 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.025083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:56.028027 (+ 2944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:56.028049 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.029851 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.029864 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:56.030119 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:56.030177 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 16:41:56.030279 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 16:41:56.030488 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":405,"thread_start_us":642,"threads_started":1}
I20260928 16:41:56.032287 29427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c39ee7f78741858df39d1dd971d6fc" candidate_uuid: "afe51739221e489ab5e6d60607fa11c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55b37ee5246744c7bed1f8579a81c222" is_pre_election: true
I20260928 16:41:56.032284 29296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c39ee7f78741858df39d1dd971d6fc" candidate_uuid: "afe51739221e489ab5e6d60607fa11c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58848017efdd433092411764a9c2157b" is_pre_election: true
I20260928 16:41:56.032882 29427 raft_consensus.cc:2454] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe51739221e489ab5e6d60607fa11c3 in term 0.
I20260928 16:41:56.032882 29296 raft_consensus.cc:2454] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe51739221e489ab5e6d60607fa11c3 in term 0.
I20260928 16:41:56.033567 29099 leader_election.cc:302] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58848017efdd433092411764a9c2157b, afe51739221e489ab5e6d60607fa11c3; no voters:
I20260928 16:41:56.034029 29499 raft_consensus.cc:2781] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:41:56.034210 29499 raft_consensus.cc:482] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:41:56.034345 29499 raft_consensus.cc:3037] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:56.036854 29499 raft_consensus.cc:504] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } }
I20260928 16:41:56.037515 29499 leader_election.cc:288] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [CANDIDATE]: Term 1 election: Requested vote from peers 55b37ee5246744c7bed1f8579a81c222 (127.10.247.195:39859), 58848017efdd433092411764a9c2157b (127.10.247.194:44591)
I20260928 16:41:56.038062 29296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c39ee7f78741858df39d1dd971d6fc" candidate_uuid: "afe51739221e489ab5e6d60607fa11c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58848017efdd433092411764a9c2157b"
I20260928 16:41:56.038069 29427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c39ee7f78741858df39d1dd971d6fc" candidate_uuid: "afe51739221e489ab5e6d60607fa11c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55b37ee5246744c7bed1f8579a81c222"
I20260928 16:41:56.038269 29296 raft_consensus.cc:3037] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:56.038267 29427 raft_consensus.cc:3037] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:56.040727 29296 raft_consensus.cc:2454] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe51739221e489ab5e6d60607fa11c3 in term 1.
I20260928 16:41:56.040755 29427 raft_consensus.cc:2454] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe51739221e489ab5e6d60607fa11c3 in term 1.
I20260928 16:41:56.041200 29100 leader_election.cc:302] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 55b37ee5246744c7bed1f8579a81c222, 58848017efdd433092411764a9c2157b, afe51739221e489ab5e6d60607fa11c3; no voters:
I20260928 16:41:56.041548 29499 raft_consensus.cc:2781] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:41:56.042032 29499 raft_consensus.cc:686] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 1 LEADER]: Becoming Leader. State: Replica: afe51739221e489ab5e6d60607fa11c3, State: Running, Role: LEADER
I20260928 16:41:56.042657 29499 consensus_queue.cc:237] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } }
I20260928 16:41:56.048255 29024 catalog_manager.cc:5495] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to afe51739221e489ab5e6d60607fa11c3 (127.10.247.193). New cstate: current_term: 1 leader_uuid: "afe51739221e489ab5e6d60607fa11c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afe51739221e489ab5e6d60607fa11c3" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 37723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 } health_report { overall_health: UNKNOWN } } }
I20260928 16:41:56.115377 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.107453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.10.247.193:37723)
0928 16:41:56.107882 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:56.107889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:56.108339 (+ 450us) server_negotiation.cc:407] Connection header received
0928 16:41:56.108816 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:56.108823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:56.109021 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:56.109229 (+ 208us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:41:56.109828 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.110814 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.112593 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.113114 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.113282 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:56.113902 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:41:56.114605 (+ 703us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:41:56.114780 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:56.114867 (+ 87us) server_negotiation.cc:299] Negotiation successful
0928 16:41:56.114985 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260928 16:41:56.125838 29427 raft_consensus.cc:1261] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [term 1 FOLLOWER]: Refusing update from remote peer afe51739221e489ab5e6d60607fa11c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:41:56.125836 29296 raft_consensus.cc:1261] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [term 1 FOLLOWER]: Refusing update from remote peer afe51739221e489ab5e6d60607fa11c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:41:56.127172 29502 consensus_queue.cc:1035] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55b37ee5246744c7bed1f8579a81c222" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 39859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:41:56.127645 29499 consensus_queue.cc:1035] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58848017efdd433092411764a9c2157b" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 44591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:41:56.141258 29506 mvcc.cc:205] Tried to move back new op lower bound from 7334353781238165504 to 7334353780915412992. Current Snapshot: MvccSnapshot[applied={T|T < 7334353781238165504}]
I20260928 16:41:56.164959 29024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:41:56.165304 29024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:41:56.169757 29024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a5bea871bd94881a7c5605a40229a39: Sending DeleteTablet for 3 replicas of tablet 82c39ee7f78741858df39d1dd971d6fc
I20260928 16:41:56.171301 29407 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c39ee7f78741858df39d1dd971d6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:41:56 UTC) from {username='slave'} at 127.0.0.1:37144
I20260928 16:41:56.171301 29145 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c39ee7f78741858df39d1dd971d6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:41:56 UTC) from {username='slave'} at 127.0.0.1:45662
I20260928 16:41:56.171417 29276 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c39ee7f78741858df39d1dd971d6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:41:56 UTC) from {username='slave'} at 127.0.0.1:44408
I20260928 16:41:56.173575 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 29083
I20260928 16:41:56.184130 29519 tablet_replica.cc:330] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: stopping tablet replica
I20260928 16:41:56.184890 29519 raft_consensus.cc:2229] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:41:56.185635 29519 raft_consensus.cc:2258] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:41:56.188810 29519 ts_tablet_manager.cc:1905] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:41:56.190721 29518 tablet_replica.cc:330] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: stopping tablet replica
I20260928 16:41:56.191735 29517 tablet_replica.cc:330] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: stopping tablet replica
I20260928 16:41:56.192276 29517 raft_consensus.cc:2229] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:41:56.193009 29517 raft_consensus.cc:2258] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:41:56.194157 29518 raft_consensus.cc:2229] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:41:56.195686 29518 raft_consensus.cc:2258] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:41:56.197947 29517 ts_tablet_manager.cc:1905] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:41:56.199396 29519 ts_tablet_manager.cc:1918] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:56.199668 29519 log.cc:1196] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/82c39ee7f78741858df39d1dd971d6fc
I20260928 16:41:56.200150 29518 ts_tablet_manager.cc:1905] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:41:56.200248 29519 ts_tablet_manager.cc:1939] T 82c39ee7f78741858df39d1dd971d6fc P 55b37ee5246744c7bed1f8579a81c222: Deleting consensus metadata
I20260928 16:41:56.202693 29010 catalog_manager.cc:4841] TS 55b37ee5246744c7bed1f8579a81c222 (127.10.247.195:39859): tablet 82c39ee7f78741858df39d1dd971d6fc (table test-table [id=b5d1342488ea478c9310a19e153b468b]) successfully deleted
I20260928 16:41:56.207468 29517 ts_tablet_manager.cc:1918] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:56.207561 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.175823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.10.247.193:37723)
0928 16:41:56.176091 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:56.176101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:56.193754 (+ 17653us) server_negotiation.cc:407] Connection header received
0928 16:41:56.194136 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:56.194148 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:56.194287 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:56.194386 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:56.195294 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.196668 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.202524 (+ 5856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.203297 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.203437 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:56.205308 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:56.205324 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:56.205347 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:41:56.206394 (+ 1047us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:56.206480 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:41:56.206522 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:56.206640 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:56.206974 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 16:41:56.207139 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260928 16:41:56.208019 29517 log.cc:1196] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/82c39ee7f78741858df39d1dd971d6fc
I20260928 16:41:56.208719 29517 ts_tablet_manager.cc:1939] T 82c39ee7f78741858df39d1dd971d6fc P 58848017efdd433092411764a9c2157b: Deleting consensus metadata
I20260928 16:41:56.208986 29205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:56.211073 29009 catalog_manager.cc:4841] TS 58848017efdd433092411764a9c2157b (127.10.247.194:44591): tablet 82c39ee7f78741858df39d1dd971d6fc (table test-table [id=b5d1342488ea478c9310a19e153b468b]) successfully deleted
I20260928 16:41:56.316691 29518 ts_tablet_manager.cc:1918] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:41:56.317041 29518 log.cc:1196] T 82c39ee7f78741858df39d1dd971d6fc P afe51739221e489ab5e6d60607fa11c3: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/82c39ee7f78741858df39d1dd971d6fc
I20260928 16:41:56.317521 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 29083
I20260928 16:41:56.336237 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 29214
W20260928 16:41:56.336403 29009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.247.193:37723 (error 108)
I20260928 16:41:56.339506 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.338137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37723 (local address 127.0.0.1:45694)
0928 16:41:56.339138 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:56.339287 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":694,"threads_started":1}
W20260928 16:41:56.339782 29009 catalog_manager.cc:4584] TS afe51739221e489ab5e6d60607fa11c3 (127.10.247.193:37723): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 82c39ee7f78741858df39d1dd971d6fc: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37723: connect: Connection refused (error 111)
I20260928 16:41:56.344511 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.337198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44412 (local address 127.10.247.194:44591)
0928 16:41:56.337418 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:56.337426 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:56.337700 (+ 274us) server_negotiation.cc:407] Connection header received
0928 16:41:56.337917 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:56.337923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:56.338054 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:56.338146 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:56.338764 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.339658 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.341230 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.341678 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.341777 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:56.342851 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:56.342861 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:56.342879 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:41:56.343744 (+ 865us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:56.343809 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:41:56.343834 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:56.343923 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:56.344047 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 16:41:56.344156 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 16:41:56.345329 29336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:56.393767 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.393202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37723 (local address 127.0.0.1:45706)
0928 16:41:56.393460 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:56.393577 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260928 16:41:56.394976 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.394549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37723 (local address 127.0.0.1:45712)
0928 16:41:56.394725 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:56.394816 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 16:41:56.441727 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 29214
I20260928 16:41:56.455860 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.455238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37723 (local address 127.0.0.1:45714)
0928 16:41:56.455524 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:56.455645 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260928 16:41:56.457242 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.456792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37723 (local address 127.0.0.1:45718)
0928 16:41:56.456959 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:56.457044 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 16:41:56.459522 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 29345
I20260928 16:41:56.467329 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.460454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37146 (local address 127.10.247.195:39859)
0928 16:41:56.460722 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:56.460727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:56.460750 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:41:56.460820 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:56.460824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:56.460927 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:56.461031 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:56.461483 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.462283 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.463779 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:56.464297 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:56.464422 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:56.465670 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:56.465682 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:56.465702 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:41:56.466472 (+ 770us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:56.466538 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:41:56.466565 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:56.466668 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:56.466778 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 16:41:56.466901 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260928 16:41:56.468209 29467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:56.538090 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.537376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37723 (local address 127.0.0.1:45732)
0928 16:41:56.537713 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:56.537851 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260928 16:41:56.539587 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:56.539017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:37723 (local address 127.0.0.1:45742)
0928 16:41:56.539278 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:56.539375 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260928 16:41:56.563452 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 29345
I20260928 16:41:56.580417 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 28993
I20260928 16:41:56.581843 29054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:41:56.705057 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 28993
2026-09-28T16:41:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3550 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:41:56 dist-test-slave-9fn6 krb5kdc[29529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:41:56 dist-test-slave-9fn6 krb5kdc[29529](info): set up 2 sockets
Sep 28 16:41:56 dist-test-slave-9fn6 krb5kdc[29529](info): commencing operation
krb5kdc: starting...
W20260928 16:41:57.761273 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:41:57 dist-test-slave-9fn6 krb5kdc[29529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:41:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:41:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:41:58.089612 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:33029
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:33029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:41:58.494460 29542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:58.494923 29542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:58.495093 29542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:58.503587 29542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:41:58.503774 29542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:58.503855 29542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:58.503938 29542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:41:58.503999 29542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:41:58.515933 29542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:33029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:33029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:58.518008 29542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:58.519793 29542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:58.534464 29547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:58.534428 29548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:58.534428 29550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:58.535679 29542 server_base.cc:1034] running on GCE node
I20260928 16:41:58.536670 29542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:58.538587 29542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:58.539825 29542 hybrid_clock.cc:648] HybridClock initialized: now 1790613718539766 us; error 46 us; skew 500 ppm
I20260928 16:41:58.540222 29542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:58 dist-test-slave-9fn6 krb5kdc[29529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613718, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:58.544085 29542 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:41:58.546034 29542 webserver.cc:460] Webserver started at http://127.10.247.254:43753/ using document root <none> and password file <none>
I20260928 16:41:58.546901 29542 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:58.547030 29542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:58.547374 29542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:58.551200 29542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3"
format_stamp: "Formatted at 2026-09-28 16:41:58 on dist-test-slave-9fn6"
server_key: "4cc859230919b132d99b73f0030d22ee"
server_key_iv: "c6c07a5afeec866b77ce9e80e39f729a"
server_key_version: "encryptionkey@0"
I20260928 16:41:58.552156 29542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3"
format_stamp: "Formatted at 2026-09-28 16:41:58 on dist-test-slave-9fn6"
server_key: "4cc859230919b132d99b73f0030d22ee"
server_key_iv: "c6c07a5afeec866b77ce9e80e39f729a"
server_key_version: "encryptionkey@0"
I20260928 16:41:58.559032 29542 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260928 16:41:58.563416 29557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:58.565129 29542 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:41:58.565315 29542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3"
format_stamp: "Formatted at 2026-09-28 16:41:58 on dist-test-slave-9fn6"
server_key: "4cc859230919b132d99b73f0030d22ee"
server_key_iv: "c6c07a5afeec866b77ce9e80e39f729a"
server_key_version: "encryptionkey@0"
I20260928 16:41:58.565558 29542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:58.578598 29542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:58.583480 29542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:58.583823 29542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:58.604012 29542 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:33029
I20260928 16:41:58.604012 29608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:33029 every 8 connection(s)
I20260928 16:41:58.605912 29542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:41:58.610934 29609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:41:58.614625 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 29542
I20260928 16:41:58.614804 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:41:58.615319 11231 external_mini_cluster.cc:1428] Setting key 66e2730923339b18f3b159da292708c4
I20260928 16:41:58.624768 29609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3: Bootstrap starting.
I20260928 16:41:58.628777 29609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3: Neither blocks nor log segments found. Creating new log.
I20260928 16:41:58.630110 29609 log.cc:824] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3: Log is configured to *not* fsync() on all Append() calls
Sep 28 16:41:58 dist-test-slave-9fn6 krb5kdc[29529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:41:58.634044 29609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3: No bootstrap required, opened a new log
I20260928 16:41:58.642030 29612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:58.617724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.10.247.254:33029)
0928 16:41:58.618651 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:58.618674 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:58.618729 (+ 55us) server_negotiation.cc:407] Connection header received
0928 16:41:58.620744 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:58.620775 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:58.621326 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:58.621806 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:58.623539 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:58.625112 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:58.626585 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:58.627396 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:58.627588 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:58.632742 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:58.632755 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:58.632776 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:58.632828 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:58.636326 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:58.637455 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:58.637465 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:58.637475 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:58.637702 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:58.638224 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:58.638229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:58.638231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:58.638963 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:58.639319 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:58.639909 (+ 590us) server_negotiation.cc:299] Negotiation successful
0928 16:41:58.640511 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":405,"threads_started":1}
I20260928 16:41:58.643261 29609 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33029 } }
I20260928 16:41:58.643762 29609 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:41:58.643906 29609 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee619f5eb0a94ceda43ddbc213dcc2b3, State: Initialized, Role: FOLLOWER
I20260928 16:41:58.644759 29609 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33029 } }
I20260928 16:41:58.645112 29609 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:41:58.645259 29609 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:41:58.645467 29609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:41:58.648149 29609 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33029 } }
I20260928 16:41:58.648804 29609 leader_election.cc:302] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee619f5eb0a94ceda43ddbc213dcc2b3; no voters:
I20260928 16:41:58.649523 29609 leader_election.cc:288] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:41:58.649663 29614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:41:58.650418 29614 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [term 1 LEADER]: Becoming Leader. State: Replica: ee619f5eb0a94ceda43ddbc213dcc2b3, State: Running, Role: LEADER
I20260928 16:41:58.651090 29614 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33029 } }
I20260928 16:41:58.651935 29609 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:41:58.654841 29616 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee619f5eb0a94ceda43ddbc213dcc2b3. Latest consensus state: current_term: 1 leader_uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33029 } } }
I20260928 16:41:58.654835 29615 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee619f5eb0a94ceda43ddbc213dcc2b3" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33029 } } }
I20260928 16:41:58.655395 29615 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:58.655395 29616 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3 [sys.catalog]: This master's current role is: LEADER
I20260928 16:41:58.657593 29620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:41:58.664229 29620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:41:58.676141 29620 catalog_manager.cc:1347] Generated new cluster ID: bf78247aa1bf48c5a6e817cea8839ab3
I20260928 16:41:58.676312 29620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:41:58.693302 29620 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:41:58.695012 29620 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:41:58.705453 29620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3: Generated new TSK 0
I20260928 16:41:58.706756 29620 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:41:58.795794 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33029
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:41:59.249359 29635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:59.249759 29635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:59.249899 29635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:59.258220 29635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:59.258376 29635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:59.258543 29635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:41:59.270721 29635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:59.272588 29635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:59.274459 29635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:59.291152 29640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:59.291177 29641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:59.291505 29643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:59.292405 29635 server_base.cc:1034] running on GCE node
I20260928 16:41:59.293239 29635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:59.294466 29635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:59.295715 29635 hybrid_clock.cc:648] HybridClock initialized: now 1790613719295642 us; error 53 us; skew 500 ppm
I20260928 16:41:59.296089 29635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:59 dist-test-slave-9fn6 krb5kdc[29529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613719, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:59.305442 29635 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:41:59.307567 29635 webserver.cc:460] Webserver started at http://127.10.247.193:42431/ using document root <none> and password file <none>
I20260928 16:41:59.308506 29635 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:59.308691 29635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:59.309078 29635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:59.312841 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "70b9205319294a8d9de8caa23f871d5a"
format_stamp: "Formatted at 2026-09-28 16:41:59 on dist-test-slave-9fn6"
server_key: "e64d5300caffb6f9275665152bf61a89"
server_key_iv: "3f4770e93fdcee2bb5405bcef51c7df3"
server_key_version: "encryptionkey@0"
I20260928 16:41:59.313817 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "70b9205319294a8d9de8caa23f871d5a"
format_stamp: "Formatted at 2026-09-28 16:41:59 on dist-test-slave-9fn6"
server_key: "e64d5300caffb6f9275665152bf61a89"
server_key_iv: "3f4770e93fdcee2bb5405bcef51c7df3"
server_key_version: "encryptionkey@0"
I20260928 16:41:59.320561 29635 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:41:59.325076 29650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:59.326930 29635 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:41:59.327129 29635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "70b9205319294a8d9de8caa23f871d5a"
format_stamp: "Formatted at 2026-09-28 16:41:59 on dist-test-slave-9fn6"
server_key: "e64d5300caffb6f9275665152bf61a89"
server_key_iv: "3f4770e93fdcee2bb5405bcef51c7df3"
server_key_version: "encryptionkey@0"
I20260928 16:41:59.327363 29635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:59.342935 29635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:59.347919 29635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:59.348296 29635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:59.349502 29635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:59.351118 29635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:59.351200 29635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:59.351334 29635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:59.351392 29635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:59.386626 29635 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:36675
I20260928 16:41:59.386660 29762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:36675 every 8 connection(s)
I20260928 16:41:59.388310 29635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:41:59.395917 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 29635
I20260928 16:41:59.396163 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:41:59.396680 11231 external_mini_cluster.cc:1428] Setting key cc67792ae0d59cd30d7c4f3f01dc30a3
Sep 28 16:41:59 dist-test-slave-9fn6 krb5kdc[29529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613719, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:41:59.412354 29765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:59.391512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:39035 (local address 127.10.247.254:33029)
0928 16:41:59.392060 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:59.392066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:59.393273 (+ 1207us) server_negotiation.cc:407] Connection header received
0928 16:41:59.395428 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:41:59.395434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:41:59.395542 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:41:59.395655 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:41:59.397675 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:59.398771 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:59.400599 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:41:59.401101 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:41:59.401234 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:59.405454 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:41:59.405459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:41:59.405463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:41:59.405506 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:41:59.408556 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:59.409489 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:59.409494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:59.409503 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:59.409660 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:41:59.410152 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:41:59.410156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:41:59.410158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:41:59.410494 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:41:59.410657 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 16:41:59.411598 (+ 941us) server_negotiation.cc:299] Negotiation successful
0928 16:41:59.411913 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":240,"threads_started":1}
I20260928 16:41:59.413745 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:59.392214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:33029 (local address 127.10.247.193:39035)
0928 16:41:59.393091 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:59.393138 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 16:41:59.395089 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:41:59.395948 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:41:59.395958 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:41:59.396692 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:41:59.397449 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:59.397468 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:59.398942 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:59.398947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:59.400415 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:41:59.400430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:41:59.401385 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:41:59.401393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:41:59.401683 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:41:59.402471 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:41:59.402507 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:41:59.405180 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:41:59.408736 (+ 3556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:59.408748 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:59.408773 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:59.409323 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:59.409779 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:41:59.409784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:41:59.409788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:41:59.410018 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:41:59.410633 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:41:59.410643 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:41:59.411134 (+ 491us) client_negotiation.cc:769] Sending connection context
0928 16:41:59.411435 (+ 301us) client_negotiation.cc:240] Negotiation successful
0928 16:41:59.412186 (+ 751us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":343,"threads_started":1}
I20260928 16:41:59.415683 29763 heartbeater.cc:347] Connected to a master server at 127.10.247.254:33029
I20260928 16:41:59.416115 29763 heartbeater.cc:464] Registering TS with master...
I20260928 16:41:59.417081 29763 heartbeater.cc:511] Master 127.10.247.254:33029 requested a full tablet report, sending...
I20260928 16:41:59.419586 29574 ts_manager.cc:194] Registered new tserver with Master: 70b9205319294a8d9de8caa23f871d5a (127.10.247.193:36675)
I20260928 16:41:59.421402 29574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:39035
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:41:59.465082 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33029
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:41:59.863153 29769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:41:59.863529 29769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:41:59.863659 29769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:41:59.871758 29769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:41:59.871906 29769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:41:59.872082 29769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:41:59.883297 29769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:41:59.885165 29769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:41:59.886873 29769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:41:59.902297 29774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:59.902208 29777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:41:59.902221 29775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:41:59.902976 29769 server_base.cc:1034] running on GCE node
I20260928 16:41:59.903716 29769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:41:59.904883 29769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:41:59.906131 29769 hybrid_clock.cc:648] HybridClock initialized: now 1790613719906062 us; error 53 us; skew 500 ppm
I20260928 16:41:59.906518 29769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:41:59 dist-test-slave-9fn6 krb5kdc[29529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613719, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:41:59.910348 29769 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:41:59.912109 29769 webserver.cc:460] Webserver started at http://127.10.247.194:34089/ using document root <none> and password file <none>
I20260928 16:41:59.913074 29769 fs_manager.cc:359] Metadata directory not provided
I20260928 16:41:59.913218 29769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:41:59.913561 29769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:41:59.917243 29769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "d51cb40ecd9c43438ee0582e169d696e"
format_stamp: "Formatted at 2026-09-28 16:41:59 on dist-test-slave-9fn6"
server_key: "c59c9bb86e9cc39074e089f69bc8229d"
server_key_iv: "acfb97b1a8068a92618f7f5fa5851345"
server_key_version: "encryptionkey@0"
I20260928 16:41:59.918200 29769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "d51cb40ecd9c43438ee0582e169d696e"
format_stamp: "Formatted at 2026-09-28 16:41:59 on dist-test-slave-9fn6"
server_key: "c59c9bb86e9cc39074e089f69bc8229d"
server_key_iv: "acfb97b1a8068a92618f7f5fa5851345"
server_key_version: "encryptionkey@0"
I20260928 16:41:59.924695 29769 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:41:59.929136 29784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:59.931401 29769 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:41:59.931639 29769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "d51cb40ecd9c43438ee0582e169d696e"
format_stamp: "Formatted at 2026-09-28 16:41:59 on dist-test-slave-9fn6"
server_key: "c59c9bb86e9cc39074e089f69bc8229d"
server_key_iv: "acfb97b1a8068a92618f7f5fa5851345"
server_key_version: "encryptionkey@0"
I20260928 16:41:59.931957 29769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:41:59.949710 29769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:41:59.954854 29769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:41:59.955163 29769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:41:59.956334 29769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:41:59.958087 29769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:41:59.958181 29769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:59.958312 29769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:41:59.958370 29769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:41:59.992805 29769 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:34513
I20260928 16:41:59.992844 29896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:34513 every 8 connection(s)
I20260928 16:41:59.994395 29769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:41:59.999658 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 29769
I20260928 16:41:59.999920 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:42:00.000461 11231 external_mini_cluster.cc:1428] Setting key efb6b19244b6e9ba5ecaa3dcb1e208b7
Sep 28 16:42:00 dist-test-slave-9fn6 krb5kdc[29529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613719, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:00.020819 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:59.997608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:45129 (local address 127.10.247.254:33029)
0928 16:41:59.998167 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0928 16:41:59.998173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:41:59.999439 (+ 1266us) server_negotiation.cc:407] Connection header received
0928 16:42:00.002318 (+ 2879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.002325 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.002432 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.002547 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:00.004842 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.005745 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.007656 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.008202 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.008341 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.013032 (+ 4691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:00.013039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:00.013043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:00.013093 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:00.016244 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:00.017549 (+ 1305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:00.017555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:00.017558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:00.017736 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:00.018400 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:00.018406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:00.018409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:00.018831 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:00.018983 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.020122 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.020410 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":283,"threads_started":1}
I20260928 16:42:00.022589 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:41:59.998181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:33029 (local address 127.10.247.194:45129)
0928 16:41:59.999179 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0928 16:41:59.999248 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:00.001895 (+ 2647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:00.002945 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:00.002961 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:00.003728 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:00.004573 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.004602 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.005940 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.005944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.007476 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.007491 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.008539 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.008548 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.008892 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.009717 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:00.009747 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:00.012705 (+ 2958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:00.016476 (+ 3771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:00.016488 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:00.016514 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:00.017324 (+ 810us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:00.017858 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:00.017863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:00.017866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:00.018186 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:00.018972 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:00.018982 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:00.019610 (+ 628us) client_negotiation.cc:769] Sending connection context
0928 16:42:00.019951 (+ 341us) client_negotiation.cc:240] Negotiation successful
0928 16:42:00.020701 (+ 750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":358,"threads_started":1}
I20260928 16:42:00.024737 29897 heartbeater.cc:347] Connected to a master server at 127.10.247.254:33029
I20260928 16:42:00.025173 29897 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:00.026216 29897 heartbeater.cc:511] Master 127.10.247.254:33029 requested a full tablet report, sending...
I20260928 16:42:00.028342 29574 ts_manager.cc:194] Registered new tserver with Master: d51cb40ecd9c43438ee0582e169d696e (127.10.247.194:34513)
I20260928 16:42:00.029434 29574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:45129
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:42:00.072459 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33029
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:42:00.425354 29763 heartbeater.cc:503] Master 127.10.247.254:33029 was elected leader, sending a full tablet report...
W20260928 16:42:00.485498 29903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:00.485884 29903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:00.486038 29903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:00.493888 29903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:00.494040 29903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:00.494215 29903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:42:00.505326 29903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:33029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:00.507153 29903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:00.508863 29903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:00.524845 29909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:00.524806 29908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:00.524844 29911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:00.526015 29903 server_base.cc:1034] running on GCE node
I20260928 16:42:00.526844 29903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:00.528146 29903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:00.529408 29903 hybrid_clock.cc:648] HybridClock initialized: now 1790613720529335 us; error 65 us; skew 500 ppm
I20260928 16:42:00.529827 29903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:00 dist-test-slave-9fn6 krb5kdc[29529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613720, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:00.533906 29903 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:42:00.535760 29903 webserver.cc:460] Webserver started at http://127.10.247.195:44565/ using document root <none> and password file <none>
I20260928 16:42:00.536718 29903 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:00.536885 29903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:00.537276 29903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:00.541208 29903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "1f960702ea064f92ba532f423657b6ef"
format_stamp: "Formatted at 2026-09-28 16:42:00 on dist-test-slave-9fn6"
server_key: "cff9df58eb723ac2923d871cb661502f"
server_key_iv: "57411088f61d1b12d5dc4d46c63b2256"
server_key_version: "encryptionkey@0"
I20260928 16:42:00.542333 29903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "1f960702ea064f92ba532f423657b6ef"
format_stamp: "Formatted at 2026-09-28 16:42:00 on dist-test-slave-9fn6"
server_key: "cff9df58eb723ac2923d871cb661502f"
server_key_iv: "57411088f61d1b12d5dc4d46c63b2256"
server_key_version: "encryptionkey@0"
I20260928 16:42:00.549268 29903 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260928 16:42:00.553848 29918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:00.555673 29903 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:42:00.555912 29903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "1f960702ea064f92ba532f423657b6ef"
format_stamp: "Formatted at 2026-09-28 16:42:00 on dist-test-slave-9fn6"
server_key: "cff9df58eb723ac2923d871cb661502f"
server_key_iv: "57411088f61d1b12d5dc4d46c63b2256"
server_key_version: "encryptionkey@0"
I20260928 16:42:00.556161 29903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:00.568758 29903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:00.573809 29903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:00.574137 29903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:00.575325 29903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:00.576963 29903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:00.577047 29903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:00.577168 29903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:00.577222 29903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:00.612103 29903 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:35589
I20260928 16:42:00.612133 30030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:35589 every 8 connection(s)
I20260928 16:42:00.614069 29903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:42:00.617268 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 29903
I20260928 16:42:00.617508 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:42:00.618167 11231 external_mini_cluster.cc:1428] Setting key e5d3f572c15810e8b817ad369c4b7a05
Sep 28 16:42:00 dist-test-slave-9fn6 krb5kdc[29529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613720, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:00.638495 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.617292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:60931 (local address 127.10.247.254:33029)
0928 16:42:00.619975 (+ 2683us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:00.619981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:00.620002 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:42:00.621959 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.621967 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.622072 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.622186 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:00.623930 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.624927 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.626634 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.627084 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.627193 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.631397 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:00.631403 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:00.631407 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:00.631451 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:00.634659 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:00.635674 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:00.635679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:00.635682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:00.635829 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:00.636342 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:00.636346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:00.636348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:00.636785 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:00.636942 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.637809 (+ 867us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.638124 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2554,"thread_start_us":2392,"threads_started":1}
I20260928 16:42:00.639642 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.618097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:33029 (local address 127.10.247.195:60931)
0928 16:42:00.619161 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:00.619234 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:00.621613 (+ 2379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:00.622407 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:00.622416 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:00.623044 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:00.623746 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.623766 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.625116 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.625120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.626479 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.626493 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.627331 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.627336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.627576 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.628355 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:00.628383 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:00.631093 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:00.634876 (+ 3783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:00.634886 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:00.634906 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:00.635548 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:00.635930 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:00.635934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:00.635942 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:00.636214 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:00.636908 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:00.636916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:00.637411 (+ 495us) client_negotiation.cc:769] Sending connection context
0928 16:42:00.637696 (+ 285us) client_negotiation.cc:240] Negotiation successful
0928 16:42:00.638255 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":370,"threads_started":1}
I20260928 16:42:00.641387 30031 heartbeater.cc:347] Connected to a master server at 127.10.247.254:33029
I20260928 16:42:00.641750 30031 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:00.642613 30031 heartbeater.cc:511] Master 127.10.247.254:33029 requested a full tablet report, sending...
I20260928 16:42:00.644394 29574 ts_manager.cc:194] Registered new tserver with Master: 1f960702ea064f92ba532f423657b6ef (127.10.247.195:35589)
I20260928 16:42:00.645476 29574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:60931
I20260928 16:42:00.647737 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:42:00.667622 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.652469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36410 (local address 127.10.247.254:33029)
0928 16:42:00.652738 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:00.652745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:00.653235 (+ 490us) server_negotiation.cc:407] Connection header received
0928 16:42:00.653459 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.653465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.653565 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.653656 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:00.655273 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.656087 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.657559 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.658027 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.658157 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.661210 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:00.661215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:00.661219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:00.661261 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:00.664197 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:00.665259 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:00.665265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:00.665267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:00.665423 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:00.665918 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:00.665922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:00.665924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:00.666321 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:00.666622 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.667047 (+ 425us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.667287 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:42:00.672677 29574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:42:00.680574 29574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:42:00.707085 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.695765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.10.247.193:36675)
0928 16:42:00.698207 (+ 2442us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:00.698215 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:00.698244 (+ 29us) server_negotiation.cc:407] Connection header received
0928 16:42:00.698355 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.698362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.698599 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.699167 (+ 568us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:00.700258 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.701711 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.704598 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.705852 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.706061 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.706279 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.706518 (+ 239us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.706705 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2246,"thread_start_us":1667,"threads_started":1}
I20260928 16:42:00.707420 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.695459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:35589 (local address 127.0.0.1:54164)
0928 16:42:00.697527 (+ 2068us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:00.697556 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:00.697779 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:00.698850 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:00.698858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:00.698908 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:00.699636 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.699674 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.701900 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.701905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.704152 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.704164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.706411 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.706418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.706634 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.706670 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 16:42:00.706746 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 16:42:00.706900 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1936,"mutex_wait_us":39,"spinlock_wait_cycles":5760,"thread_start_us":1932,"threads_started":1}
I20260928 16:42:00.707752 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.695916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54164 (local address 127.10.247.195:35589)
0928 16:42:00.697223 (+ 1307us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:00.697233 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:00.697666 (+ 433us) server_negotiation.cc:407] Connection header received
0928 16:42:00.697972 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.697981 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.698281 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.698460 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:00.699882 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.701755 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.704317 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.705892 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.706089 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.707141 (+ 1052us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.707215 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.707383 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1128,"thread_start_us":962,"threads_started":1}
I20260928 16:42:00.708115 30044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.695329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:36675 (local address 127.0.0.1:53852)
0928 16:42:00.696221 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:00.696281 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:00.696498 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:00.698850 (+ 2352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:00.698858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:00.698900 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:00.700088 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.700099 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.701897 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.701902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.704404 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.704419 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.706051 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.706061 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.706245 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.706296 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 16:42:00.706439 (+ 143us) client_negotiation.cc:240] Negotiation successful
0928 16:42:00.706680 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":46,"spinlock_wait_cycles":768,"thread_start_us":415,"threads_started":1}
I20260928 16:42:00.710750 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.697877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34513 (local address 127.0.0.1:35252)
0928 16:42:00.699918 (+ 2041us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:00.699938 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:00.700066 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:00.702512 (+ 2446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:00.702517 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:00.702545 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:00.703106 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.703119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.705300 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.705305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.707399 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.707420 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.709964 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.709971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.710155 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.710191 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 16:42:00.710288 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 16:42:00.710440 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1908,"thread_start_us":1687,"threads_started":1}
I20260928 16:42:00.710876 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.698428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.10.247.194:34513)
0928 16:42:00.701886 (+ 3458us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:00.701894 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:00.701920 (+ 26us) server_negotiation.cc:407] Connection header received
0928 16:42:00.702028 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.702036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.702292 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.702498 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:00.703287 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.705122 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.707590 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.709738 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.709910 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.710091 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.710349 (+ 258us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.710553 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3286,"thread_start_us":3143,"threads_started":1}
I20260928 16:42:00.712548 29698 tablet_service.cc:1467] Processing CreateTablet for tablet 4cfe0424c246496d8e55d3e70566b354 (DEFAULT_TABLE table=test-table [id=bb8a577ea2034b939d55ccb290d24127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:00.712566 29966 tablet_service.cc:1467] Processing CreateTablet for tablet 4cfe0424c246496d8e55d3e70566b354 (DEFAULT_TABLE table=test-table [id=bb8a577ea2034b939d55ccb290d24127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:00.714360 29966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cfe0424c246496d8e55d3e70566b354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:00.714347 29698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cfe0424c246496d8e55d3e70566b354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:00.715305 29832 tablet_service.cc:1467] Processing CreateTablet for tablet 4cfe0424c246496d8e55d3e70566b354 (DEFAULT_TABLE table=test-table [id=bb8a577ea2034b939d55ccb290d24127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:00.716919 29832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cfe0424c246496d8e55d3e70566b354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:00.730034 30050 tablet_bootstrap.cc:492] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: Bootstrap starting.
I20260928 16:42:00.732995 30052 tablet_bootstrap.cc:492] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: Bootstrap starting.
I20260928 16:42:00.735338 30050 tablet_bootstrap.cc:654] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:00.735877 30051 tablet_bootstrap.cc:492] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: Bootstrap starting.
I20260928 16:42:00.737418 30052 tablet_bootstrap.cc:654] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:00.737506 30050 log.cc:824] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:00.738904 30052 log.cc:824] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:00.740191 30051 tablet_bootstrap.cc:654] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:00.741405 30051 log.cc:824] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:00.742950 30052 tablet_bootstrap.cc:492] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: No bootstrap required, opened a new log
I20260928 16:42:00.743290 30052 ts_tablet_manager.cc:1397] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260928 16:42:00.744356 30050 tablet_bootstrap.cc:492] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: No bootstrap required, opened a new log
I20260928 16:42:00.744527 30051 tablet_bootstrap.cc:492] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: No bootstrap required, opened a new log
I20260928 16:42:00.744666 30050 ts_tablet_manager.cc:1397] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260928 16:42:00.744819 30051 ts_tablet_manager.cc:1397] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260928 16:42:00.753324 30050 raft_consensus.cc:348] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } }
I20260928 16:42:00.753854 30050 raft_consensus.cc:374] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:00.753968 30050 raft_consensus.cc:729] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70b9205319294a8d9de8caa23f871d5a, State: Initialized, Role: FOLLOWER
I20260928 16:42:00.754746 30050 consensus_queue.cc:260] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } }
I20260928 16:42:00.754922 30052 raft_consensus.cc:348] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } }
I20260928 16:42:00.755429 30052 raft_consensus.cc:374] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:00.755534 30052 raft_consensus.cc:729] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51cb40ecd9c43438ee0582e169d696e, State: Initialized, Role: FOLLOWER
I20260928 16:42:00.755927 30051 raft_consensus.cc:348] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } }
I20260928 16:42:00.756147 30052 consensus_queue.cc:260] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } }
I20260928 16:42:00.756390 30051 raft_consensus.cc:374] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:00.756474 30051 raft_consensus.cc:729] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f960702ea064f92ba532f423657b6ef, State: Initialized, Role: FOLLOWER
I20260928 16:42:00.757215 30051 consensus_queue.cc:260] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } }
I20260928 16:42:00.757680 29897 heartbeater.cc:503] Master 127.10.247.254:33029 was elected leader, sending a full tablet report...
I20260928 16:42:00.758744 30052 ts_tablet_manager.cc:1428] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: Time spent starting tablet: real 0.015s user 0.014s sys 0.004s
I20260928 16:42:00.761407 30031 heartbeater.cc:503] Master 127.10.247.254:33029 was elected leader, sending a full tablet report...
I20260928 16:42:00.762472 30051 ts_tablet_manager.cc:1428] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
I20260928 16:42:00.764200 30050 ts_tablet_manager.cc:1428] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: Time spent starting tablet: real 0.019s user 0.008s sys 0.010s
I20260928 16:42:00.814297 30056 raft_consensus.cc:482] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:42:00.814684 30056 raft_consensus.cc:504] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } }
I20260928 16:42:00.816985 30056 leader_election.cc:288] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f960702ea064f92ba532f423657b6ef (127.10.247.195:35589), d51cb40ecd9c43438ee0582e169d696e (127.10.247.194:34513)
I20260928 16:42:00.824189 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.817619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:36125 (local address 127.10.247.195:35589)
0928 16:42:00.817899 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:00.817906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:00.818182 (+ 276us) server_negotiation.cc:407] Connection header received
0928 16:42:00.818509 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.818515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.818606 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.818681 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:00.819559 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.820387 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.822337 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.823159 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.823316 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.823403 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.823765 (+ 362us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.823906 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260928 16:42:00.824208 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.818403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:34915 (local address 127.10.247.194:34513)
0928 16:42:00.818644 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:00.818650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:00.818887 (+ 237us) server_negotiation.cc:407] Connection header received
0928 16:42:00.819018 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.819024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.819106 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.819194 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:00.819788 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.820546 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.822337 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.823183 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.823326 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.823408 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.823773 (+ 365us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.823919 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260928 16:42:00.824638 30059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.817292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:35589 (local address 127.10.247.193:36125)
0928 16:42:00.818031 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:00.818082 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:00.818291 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:00.818918 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:00.818923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:00.818952 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:00.819397 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.819412 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.820524 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.820530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.822182 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.822194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.823393 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.823398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.823593 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.823633 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 16:42:00.823717 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 16:42:00.823882 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":28,"thread_start_us":354,"threads_started":1}
I20260928 16:42:00.824708 30060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.818091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34513 (local address 127.10.247.193:34915)
0928 16:42:00.818770 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:00.818788 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:00.818907 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:00.819228 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:00.819231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:00.819250 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:00.819625 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.819633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.820707 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.820711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.822173 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:00.822186 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.823430 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.823437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:00.823607 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.823648 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 16:42:00.823725 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 16:42:00.823872 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":33,"thread_start_us":326,"threads_started":1}
I20260928 16:42:00.826081 29986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cfe0424c246496d8e55d3e70566b354" candidate_uuid: "70b9205319294a8d9de8caa23f871d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f960702ea064f92ba532f423657b6ef" is_pre_election: true
I20260928 16:42:00.826084 29852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cfe0424c246496d8e55d3e70566b354" candidate_uuid: "70b9205319294a8d9de8caa23f871d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51cb40ecd9c43438ee0582e169d696e" is_pre_election: true
I20260928 16:42:00.826766 29986 raft_consensus.cc:2454] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70b9205319294a8d9de8caa23f871d5a in term 0.
I20260928 16:42:00.826766 29852 raft_consensus.cc:2454] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70b9205319294a8d9de8caa23f871d5a in term 0.
I20260928 16:42:00.827610 29653 leader_election.cc:302] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f960702ea064f92ba532f423657b6ef, 70b9205319294a8d9de8caa23f871d5a; no voters:
I20260928 16:42:00.828119 30056 raft_consensus.cc:2781] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:42:00.828298 30056 raft_consensus.cc:482] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:42:00.828411 30056 raft_consensus.cc:3037] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:00.831238 30056 raft_consensus.cc:504] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } }
I20260928 16:42:00.832053 30056 leader_election.cc:288] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [CANDIDATE]: Term 1 election: Requested vote from peers 1f960702ea064f92ba532f423657b6ef (127.10.247.195:35589), d51cb40ecd9c43438ee0582e169d696e (127.10.247.194:34513)
I20260928 16:42:00.832863 29986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cfe0424c246496d8e55d3e70566b354" candidate_uuid: "70b9205319294a8d9de8caa23f871d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f960702ea064f92ba532f423657b6ef"
I20260928 16:42:00.832933 29852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cfe0424c246496d8e55d3e70566b354" candidate_uuid: "70b9205319294a8d9de8caa23f871d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51cb40ecd9c43438ee0582e169d696e"
I20260928 16:42:00.833138 29986 raft_consensus.cc:3037] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:00.833235 29852 raft_consensus.cc:3037] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:00.835847 29986 raft_consensus.cc:2454] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70b9205319294a8d9de8caa23f871d5a in term 1.
I20260928 16:42:00.836010 29852 raft_consensus.cc:2454] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70b9205319294a8d9de8caa23f871d5a in term 1.
I20260928 16:42:00.836403 29653 leader_election.cc:302] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f960702ea064f92ba532f423657b6ef, 70b9205319294a8d9de8caa23f871d5a; no voters:
I20260928 16:42:00.836802 30056 raft_consensus.cc:2781] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:00.837395 30056 raft_consensus.cc:686] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 1 LEADER]: Becoming Leader. State: Replica: 70b9205319294a8d9de8caa23f871d5a, State: Running, Role: LEADER
I20260928 16:42:00.838071 30056 consensus_queue.cc:237] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } }
I20260928 16:42:00.843524 29573 catalog_manager.cc:5495] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 70b9205319294a8d9de8caa23f871d5a (127.10.247.193). New cstate: current_term: 1 leader_uuid: "70b9205319294a8d9de8caa23f871d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b9205319294a8d9de8caa23f871d5a" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 } health_report { overall_health: UNKNOWN } } }
W20260928 16:42:00.866037 30032 tablet.cc:2369] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:00.871855 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.865750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53854 (local address 127.10.247.193:36675)
0928 16:42:00.865966 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:00.865973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:00.866125 (+ 152us) server_negotiation.cc:407] Connection header received
0928 16:42:00.866343 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.866348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.866477 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.866653 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:00.867074 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.867884 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.869380 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.869818 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.869923 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.870556 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:00.871228 (+ 672us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:00.871380 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.871541 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.871597 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 16:42:00.882611 29852 raft_consensus.cc:1261] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [term 1 FOLLOWER]: Refusing update from remote peer 70b9205319294a8d9de8caa23f871d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:00.882611 29986 raft_consensus.cc:1261] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [term 1 FOLLOWER]: Refusing update from remote peer 70b9205319294a8d9de8caa23f871d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:00.883754 30061 consensus_queue.cc:1035] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f960702ea064f92ba532f423657b6ef" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 35589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:00.884169 30056 consensus_queue.cc:1035] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51cb40ecd9c43438ee0582e169d696e" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 16:42:00.891239 29764 tablet.cc:2369] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:00.897467 30064 mvcc.cc:205] Tried to move back new op lower bound from 7334353800721207296 to 7334353800557334528. Current Snapshot: MvccSnapshot[applied={T|T < 7334353800721207296}]
I20260928 16:42:00.914976 29574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:42:00.915329 29574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:42:00.920027 29574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee619f5eb0a94ceda43ddbc213dcc2b3: Sending DeleteTablet for 3 replicas of tablet 4cfe0424c246496d8e55d3e70566b354
I20260928 16:42:00.921689 29966 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cfe0424c246496d8e55d3e70566b354 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:00 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:54164
I20260928 16:42:00.921687 29698 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cfe0424c246496d8e55d3e70566b354 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:00 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:53852
I20260928 16:42:00.921849 29832 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cfe0424c246496d8e55d3e70566b354 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:00 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:35252
I20260928 16:42:00.924783 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 29635
I20260928 16:42:00.933100 30074 tablet_replica.cc:330] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: stopping tablet replica
I20260928 16:42:00.934057 30074 raft_consensus.cc:2229] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:42:00.934903 30075 tablet_replica.cc:330] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: stopping tablet replica
I20260928 16:42:00.935835 30074 raft_consensus.cc:2258] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:00.936625 30075 raft_consensus.cc:2229] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:42:00.939100 30075 raft_consensus.cc:2258] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:00.941076 30074 ts_tablet_manager.cc:1905] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:00.944331 30075 ts_tablet_manager.cc:1905] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:00.951323 30076 tablet_replica.cc:330] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: stopping tablet replica
I20260928 16:42:00.953848 30075 ts_tablet_manager.cc:1918] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:00.954187 30075 log.cc:1196] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/4cfe0424c246496d8e55d3e70566b354
I20260928 16:42:00.954926 30075 ts_tablet_manager.cc:1939] T 4cfe0424c246496d8e55d3e70566b354 P 1f960702ea064f92ba532f423657b6ef: Deleting consensus metadata
Sep 28 16:42:00 dist-test-slave-9fn6 krb5kdc[29529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:42:00.955978 30076 raft_consensus.cc:2229] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:42:00.956452 30074 ts_tablet_manager.cc:1918] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:00.956818 30074 log.cc:1196] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/4cfe0424c246496d8e55d3e70566b354
I20260928 16:42:00.957516 30074 ts_tablet_manager.cc:1939] T 4cfe0424c246496d8e55d3e70566b354 P 70b9205319294a8d9de8caa23f871d5a: Deleting consensus metadata
I20260928 16:42:00.957698 29560 catalog_manager.cc:4841] TS 1f960702ea064f92ba532f423657b6ef (127.10.247.195:35589): tablet 4cfe0424c246496d8e55d3e70566b354 (table test-table [id=bb8a577ea2034b939d55ccb290d24127]) successfully deleted
I20260928 16:42:00.958248 30076 raft_consensus.cc:2258] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:00.959960 29561 catalog_manager.cc:4841] TS 70b9205319294a8d9de8caa23f871d5a (127.10.247.193:36675): tablet 4cfe0424c246496d8e55d3e70566b354 (table test-table [id=bb8a577ea2034b939d55ccb290d24127]) successfully deleted
I20260928 16:42:00.961875 30076 ts_tablet_manager.cc:1905] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:00.967693 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:00.934856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53866 (local address 127.10.247.193:36675)
0928 16:42:00.935084 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:00.935091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:00.942402 (+ 7311us) server_negotiation.cc:407] Connection header received
0928 16:42:00.943061 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:00.943068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:00.943193 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:00.943283 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:00.945486 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.946722 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.948703 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:00.949487 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:00.949603 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:00.958526 (+ 8923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:00.958545 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:00.958569 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:00.958641 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:00.963562 (+ 4921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:00.964791 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:00.964803 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:00.964807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:00.965024 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:00.965576 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:00.965583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:00.965586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:00.966201 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:00.966565 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:00.966924 (+ 359us) server_negotiation.cc:299] Negotiation successful
0928 16:42:00.967271 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:42:00.968569 29758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:00.969357 30076 ts_tablet_manager.cc:1918] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:00.969579 30076 log.cc:1196] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/4cfe0424c246496d8e55d3e70566b354
I20260928 16:42:00.970223 30076 ts_tablet_manager.cc:1939] T 4cfe0424c246496d8e55d3e70566b354 P d51cb40ecd9c43438ee0582e169d696e: Deleting consensus metadata
I20260928 16:42:00.972683 29560 catalog_manager.cc:4841] TS d51cb40ecd9c43438ee0582e169d696e (127.10.247.194:34513): tablet 4cfe0424c246496d8e55d3e70566b354 (table test-table [id=bb8a577ea2034b939d55ccb290d24127]) successfully deleted
I20260928 16:42:01.094070 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 29635
I20260928 16:42:01.113336 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 29769
Sep 28 16:42:01 dist-test-slave-9fn6 krb5kdc[29529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:42:01.132040 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:01.114297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35266 (local address 127.10.247.194:34513)
0928 16:42:01.114528 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:01.114535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:01.114634 (+ 99us) server_negotiation.cc:407] Connection header received
0928 16:42:01.114856 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:01.114861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:01.114954 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:01.115047 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:01.116534 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:01.117566 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:01.118945 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:01.119368 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:01.119481 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:01.124119 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:01.124140 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:01.124162 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:01.124210 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:01.128327 (+ 4117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:01.129446 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:01.129460 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:01.129462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:01.129684 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:01.130164 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:01.130168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:01.130171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:01.130653 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:01.130931 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:01.131331 (+ 400us) server_negotiation.cc:299] Negotiation successful
0928 16:42:01.131582 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260928 16:42:01.132899 29892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:01.232035 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 29769
I20260928 16:42:01.249178 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 29903
Sep 28 16:42:01 dist-test-slave-9fn6 krb5kdc[29529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:42:01.268034 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:01.250234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54166 (local address 127.10.247.195:35589)
0928 16:42:01.250469 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:01.250475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:01.250502 (+ 27us) server_negotiation.cc:407] Connection header received
0928 16:42:01.250642 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:01.250648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:01.250745 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:01.250845 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:01.252511 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:01.253490 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:01.254933 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:01.255402 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:01.255530 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:01.261091 (+ 5561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:01.261116 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:01.261138 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:01.261192 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:01.264453 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:01.265672 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:01.265683 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:01.265686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:01.265897 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:01.266380 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:01.266384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:01.266386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:01.266873 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:01.267164 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:01.267501 (+ 337us) server_negotiation.cc:299] Negotiation successful
0928 16:42:01.267721 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260928 16:42:01.268882 30026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:01.366569 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 29903
I20260928 16:42:01.384194 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 29542
I20260928 16:42:01.385672 29604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:01.514856 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 29542
2026-09-28T16:42:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4809 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-28T16:42:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:01Z Disabled control of system clock
I20260928 16:42:01.606140 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:43605
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:43605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:42:02.009186 30085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:02.009603 30085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:02.009745 30085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:02.017978 30085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:02.018127 30085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:02.018173 30085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:02.018239 30085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:02.018285 30085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:02.028741 30085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:43605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:43605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:02.030562 30085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:02.032831 30085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:02.046653 30090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:02.047124 30093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:02.046615 30091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:02.058810 30085 server_base.cc:1034] running on GCE node
I20260928 16:42:02.060012 30085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:02.062141 30085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:02.063401 30085 hybrid_clock.cc:648] HybridClock initialized: now 1790613722063340 us; error 59 us; skew 500 ppm
I20260928 16:42:02.063843 30085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:42:02.066781 30085 webserver.cc:460] Webserver started at http://127.10.247.254:45617/ using document root <none> and password file <none>
I20260928 16:42:02.067905 30085 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:02.068053 30085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:02.068506 30085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:02.072376 30085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "8cf3e19be6694486973750cc87b443ec"
format_stamp: "Formatted at 2026-09-28 16:42:02 on dist-test-slave-9fn6"
server_key: "c5d36aac2b6d2fbfa0840008a4bbe927"
server_key_iv: "2edcf570c9a9d90a6e4e246992b52b58"
server_key_version: "encryptionkey@0"
I20260928 16:42:02.073498 30085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "8cf3e19be6694486973750cc87b443ec"
format_stamp: "Formatted at 2026-09-28 16:42:02 on dist-test-slave-9fn6"
server_key: "c5d36aac2b6d2fbfa0840008a4bbe927"
server_key_iv: "2edcf570c9a9d90a6e4e246992b52b58"
server_key_version: "encryptionkey@0"
I20260928 16:42:02.080124 30085 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260928 16:42:02.084668 30099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:02.086450 30085 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:42:02.086663 30085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "8cf3e19be6694486973750cc87b443ec"
format_stamp: "Formatted at 2026-09-28 16:42:02 on dist-test-slave-9fn6"
server_key: "c5d36aac2b6d2fbfa0840008a4bbe927"
server_key_iv: "2edcf570c9a9d90a6e4e246992b52b58"
server_key_version: "encryptionkey@0"
I20260928 16:42:02.086939 30085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:02.111171 30085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:02.112361 30085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:02.112731 30085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:02.134949 30085 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:43605
I20260928 16:42:02.134965 30150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:43605 every 8 connection(s)
I20260928 16:42:02.136695 30085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:02.140081 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 30085
I20260928 16:42:02.140290 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:02.140862 11231 external_mini_cluster.cc:1428] Setting key eff94086014705958aae2a228e91c30d
I20260928 16:42:02.141633 30151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:02.158109 30151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec: Bootstrap starting.
I20260928 16:42:02.158548 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:02.143037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.10.247.254:43605)
0928 16:42:02.144186 (+ 1149us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:02.144200 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:02.144250 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:42:02.147036 (+ 2786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:02.147076 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:02.147735 (+ 659us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:02.148182 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:02.148981 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:02.150486 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:02.152573 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:02.153504 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:02.153673 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:02.155304 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:02.155314 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:02.155338 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:02.156113 (+ 775us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:02.156223 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:02.156250 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:02.156360 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:02.156719 (+ 359us) server_negotiation.cc:299] Negotiation successful
0928 16:42:02.157222 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":402,"threads_started":1}
I20260928 16:42:02.162104 30151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:02.163395 30151 log.cc:824] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:02.166970 30151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec: No bootstrap required, opened a new log
I20260928 16:42:02.175772 30151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf3e19be6694486973750cc87b443ec" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43605 } }
I20260928 16:42:02.176317 30151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:02.176434 30151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf3e19be6694486973750cc87b443ec, State: Initialized, Role: FOLLOWER
I20260928 16:42:02.177284 30151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf3e19be6694486973750cc87b443ec" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43605 } }
I20260928 16:42:02.177592 30151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:02.177770 30151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:02.177961 30151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:02.180600 30151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf3e19be6694486973750cc87b443ec" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43605 } }
I20260928 16:42:02.181378 30151 leader_election.cc:302] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cf3e19be6694486973750cc87b443ec; no voters:
I20260928 16:42:02.182217 30151 leader_election.cc:288] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:02.182413 30156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:02.183128 30156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [term 1 LEADER]: Becoming Leader. State: Replica: 8cf3e19be6694486973750cc87b443ec, State: Running, Role: LEADER
I20260928 16:42:02.183784 30156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf3e19be6694486973750cc87b443ec" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43605 } }
I20260928 16:42:02.184774 30151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:02.187378 30157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cf3e19be6694486973750cc87b443ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf3e19be6694486973750cc87b443ec" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43605 } } }
I20260928 16:42:02.188002 30157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:02.189793 30158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cf3e19be6694486973750cc87b443ec. Latest consensus state: current_term: 1 leader_uuid: "8cf3e19be6694486973750cc87b443ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf3e19be6694486973750cc87b443ec" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43605 } } }
I20260928 16:42:02.190364 30158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:02.190900 30164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:02.198073 30164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:02.210454 30164 catalog_manager.cc:1347] Generated new cluster ID: d9b0e4dcf0454034b234fea311f85bbb
I20260928 16:42:02.210659 30164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:02.230227 30164 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:02.232600 30164 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:02.247815 30164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec: Generated new TSK 0
I20260928 16:42:02.249384 30164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:02.265096 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:43605
--builtin_ntp_servers=127.10.247.212:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:42:02.665905 30175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:02.666292 30175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:02.666420 30175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:02.674633 30175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:02.674808 30175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:02.674968 30175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:42:02.685603 30175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:43605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:02.687394 30175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:02.689726 30175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:02.706061 30180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:02.705988 30181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:02.706036 30183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:02.706949 30175 server_base.cc:1034] running on GCE node
I20260928 16:42:02.707820 30175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:02.709229 30175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:02.710474 30175 hybrid_clock.cc:648] HybridClock initialized: now 1790613722710413 us; error 60 us; skew 500 ppm
I20260928 16:42:02.710872 30175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:42:02.713966 30175 webserver.cc:460] Webserver started at http://127.10.247.193:36221/ using document root <none> and password file <none>
I20260928 16:42:02.714931 30175 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:02.715057 30175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:02.715446 30175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:02.719419 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "b372e0244efd4128a3ab91e951985e11"
format_stamp: "Formatted at 2026-09-28 16:42:02 on dist-test-slave-9fn6"
server_key: "255cc4eb54e8887452977c3ca4a603ea"
server_key_iv: "0a707faa70fb51599adf05fabac5fc06"
server_key_version: "encryptionkey@0"
I20260928 16:42:02.720510 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "b372e0244efd4128a3ab91e951985e11"
format_stamp: "Formatted at 2026-09-28 16:42:02 on dist-test-slave-9fn6"
server_key: "255cc4eb54e8887452977c3ca4a603ea"
server_key_iv: "0a707faa70fb51599adf05fabac5fc06"
server_key_version: "encryptionkey@0"
I20260928 16:42:02.727692 30175 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260928 16:42:02.732267 30189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:02.734062 30175 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:42:02.734284 30175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "b372e0244efd4128a3ab91e951985e11"
format_stamp: "Formatted at 2026-09-28 16:42:02 on dist-test-slave-9fn6"
server_key: "255cc4eb54e8887452977c3ca4a603ea"
server_key_iv: "0a707faa70fb51599adf05fabac5fc06"
server_key_version: "encryptionkey@0"
I20260928 16:42:02.734556 30175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:02.774789 30175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:02.776012 30175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:02.776371 30175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:02.777694 30175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:02.779690 30175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:02.779795 30175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:02.779915 30175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:02.779997 30175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:02.814991 30175 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:36973
I20260928 16:42:02.815035 30301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:36973 every 8 connection(s)
I20260928 16:42:02.816987 30175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:42:02.820428 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 30175
I20260928 16:42:02.820670 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:42:02.821344 11231 external_mini_cluster.cc:1428] Setting key 0f76eec17ec2a25e78bd56168e8c29c0
I20260928 16:42:02.825254 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:43605
--builtin_ntp_servers=127.10.247.212:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:42:02.834420 30304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:02.820643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:39435 (local address 127.10.247.254:43605)
0928 16:42:02.821314 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:02.821322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:02.823005 (+ 1683us) server_negotiation.cc:407] Connection header received
0928 16:42:02.825746 (+ 2741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:02.825751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:02.825837 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:02.825948 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:02.827787 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:02.828717 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:02.831026 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:02.831484 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:02.831618 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:02.833500 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:02.833506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:02.833510 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:02.833536 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:02.833590 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:02.833604 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:02.833693 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:02.833940 (+ 247us) server_negotiation.cc:299] Negotiation successful
0928 16:42:02.834067 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":319,"threads_started":1}
I20260928 16:42:02.836171 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:02.821505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:43605 (local address 127.10.247.193:39435)
0928 16:42:02.822824 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:02.822875 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:02.825355 (+ 2480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:02.826282 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:02.826297 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:02.826470 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:02.827527 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:02.827567 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:02.828896 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:02.828902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:02.830844 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:02.830861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:02.831759 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:02.831766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:02.832087 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:02.833015 (+ 928us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:42:02.833019 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:02.833087 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:42:02.833227 (+ 140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:02.833704 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:02.833715 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:02.833724 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 16:42:02.833927 (+ 203us) client_negotiation.cc:240] Negotiation successful
0928 16:42:02.834449 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":362,"threads_started":1}
I20260928 16:42:02.838449 30302 heartbeater.cc:347] Connected to a master server at 127.10.247.254:43605
I20260928 16:42:02.838878 30302 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:02.839952 30302 heartbeater.cc:511] Master 127.10.247.254:43605 requested a full tablet report, sending...
I20260928 16:42:02.842865 30116 ts_manager.cc:194] Registered new tserver with Master: b372e0244efd4128a3ab91e951985e11 (127.10.247.193:36973)
I20260928 16:42:02.844686 30116 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.193:39435
W20260928 16:42:03.228345 30306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:03.228765 30306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:03.228909 30306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:03.237030 30306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:03.237170 30306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:03.237342 30306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:42:03.247728 30306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:43605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:03.249636 30306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:03.251930 30306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:03.267098 30311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:03.267098 30312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:03.267143 30314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:03.268049 30306 server_base.cc:1034] running on GCE node
I20260928 16:42:03.268774 30306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:03.269910 30306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:03.271157 30306 hybrid_clock.cc:648] HybridClock initialized: now 1790613723271110 us; error 83 us; skew 500 ppm
I20260928 16:42:03.271569 30306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:42:03.274418 30306 webserver.cc:460] Webserver started at http://127.10.247.194:40009/ using document root <none> and password file <none>
I20260928 16:42:03.275396 30306 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:03.275529 30306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:03.275900 30306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:03.279718 30306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "17c264ec094b4b17892e518159b0137d"
format_stamp: "Formatted at 2026-09-28 16:42:03 on dist-test-slave-9fn6"
server_key: "4ee0a569e7bf03df9c8d999d20580541"
server_key_iv: "5b6aa951a826ca217a683d372bf23d46"
server_key_version: "encryptionkey@0"
I20260928 16:42:03.280728 30306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "17c264ec094b4b17892e518159b0137d"
format_stamp: "Formatted at 2026-09-28 16:42:03 on dist-test-slave-9fn6"
server_key: "4ee0a569e7bf03df9c8d999d20580541"
server_key_iv: "5b6aa951a826ca217a683d372bf23d46"
server_key_version: "encryptionkey@0"
I20260928 16:42:03.287201 30306 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260928 16:42:03.291620 30320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:03.293367 30306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:42:03.293555 30306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "17c264ec094b4b17892e518159b0137d"
format_stamp: "Formatted at 2026-09-28 16:42:03 on dist-test-slave-9fn6"
server_key: "4ee0a569e7bf03df9c8d999d20580541"
server_key_iv: "5b6aa951a826ca217a683d372bf23d46"
server_key_version: "encryptionkey@0"
I20260928 16:42:03.293808 30306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:03.319936 30306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:03.321251 30306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:03.321589 30306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:03.322839 30306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:03.324724 30306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:03.324837 30306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:03.324960 30306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:03.325037 30306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:03.360031 30306 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:33085
I20260928 16:42:03.360040 30432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:33085 every 8 connection(s)
I20260928 16:42:03.361788 30306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:42:03.369817 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 30306
I20260928 16:42:03.370040 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:42:03.370569 11231 external_mini_cluster.cc:1428] Setting key 64ca8f43cd9529f5b6a7b3b70a722f6b
I20260928 16:42:03.374254 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:43605
--builtin_ntp_servers=127.10.247.212:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:42:03.377530 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.364995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:48291 (local address 127.10.247.254:43605)
0928 16:42:03.365591 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:03.365596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:03.367082 (+ 1486us) server_negotiation.cc:407] Connection header received
0928 16:42:03.369415 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:03.369422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:03.369521 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:03.369632 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:03.371079 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.372094 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.374131 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.374663 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.374790 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:03.376485 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:03.376490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:03.376494 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:03.376520 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:03.376572 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:03.376586 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:03.376728 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:03.376972 (+ 244us) server_negotiation.cc:299] Negotiation successful
0928 16:42:03.377114 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":272,"threads_started":1}
I20260928 16:42:03.378959 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.365779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:43605 (local address 127.10.247.194:48291)
0928 16:42:03.366871 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:03.366914 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:03.369048 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:03.369940 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:03.369951 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:03.370095 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:03.370879 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:03.370902 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.372277 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.372283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:03.373953 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:03.373970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.374943 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.374951 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:03.375232 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:03.376055 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:42:03.376058 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:03.376122 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:42:03.376250 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:03.376720 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:03.376734 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:03.376745 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 16:42:03.376948 (+ 203us) client_negotiation.cc:240] Negotiation successful
0928 16:42:03.377522 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":320,"threads_started":1}
I20260928 16:42:03.380949 30433 heartbeater.cc:347] Connected to a master server at 127.10.247.254:43605
I20260928 16:42:03.381366 30433 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:03.382386 30433 heartbeater.cc:511] Master 127.10.247.254:43605 requested a full tablet report, sending...
I20260928 16:42:03.384292 30116 ts_manager.cc:194] Registered new tserver with Master: 17c264ec094b4b17892e518159b0137d (127.10.247.194:33085)
I20260928 16:42:03.385581 30116 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.194:48291
W20260928 16:42:03.772281 30437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:03.772739 30437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:03.772900 30437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:03.781193 30437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:03.781339 30437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:03.781497 30437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:42:03.792117 30437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:43605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:03.793912 30437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:03.796144 30437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:03.812279 30442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:03.812217 30445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:03.812214 30443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:03.812733 30437 server_base.cc:1034] running on GCE node
I20260928 16:42:03.814045 30437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:03.821307 30437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:03.822629 30437 hybrid_clock.cc:648] HybridClock initialized: now 1790613723822547 us; error 89 us; skew 500 ppm
I20260928 16:42:03.823108 30437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:42:03.826130 30437 webserver.cc:460] Webserver started at http://127.10.247.195:44911/ using document root <none> and password file <none>
I20260928 16:42:03.827050 30437 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:03.827173 30437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:03.827538 30437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:03.831161 30437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "5f2f92d5ab9f4826ae86beca2802321d"
format_stamp: "Formatted at 2026-09-28 16:42:03 on dist-test-slave-9fn6"
server_key: "560a4af26eecf54c7e88ea5860254b84"
server_key_iv: "64b8d5fa235ab366858bee707d99c1dc"
server_key_version: "encryptionkey@0"
I20260928 16:42:03.832130 30437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "5f2f92d5ab9f4826ae86beca2802321d"
format_stamp: "Formatted at 2026-09-28 16:42:03 on dist-test-slave-9fn6"
server_key: "560a4af26eecf54c7e88ea5860254b84"
server_key_iv: "64b8d5fa235ab366858bee707d99c1dc"
server_key_version: "encryptionkey@0"
I20260928 16:42:03.838819 30437 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260928 16:42:03.843535 30451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:03.845549 30437 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260928 16:42:03.845770 30437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "5f2f92d5ab9f4826ae86beca2802321d"
format_stamp: "Formatted at 2026-09-28 16:42:03 on dist-test-slave-9fn6"
server_key: "560a4af26eecf54c7e88ea5860254b84"
server_key_iv: "64b8d5fa235ab366858bee707d99c1dc"
server_key_version: "encryptionkey@0"
I20260928 16:42:03.846043 30437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:03.848835 30302 heartbeater.cc:503] Master 127.10.247.254:43605 was elected leader, sending a full tablet report...
I20260928 16:42:03.878345 30437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:03.879529 30437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:03.879868 30437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:03.881145 30437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:03.883034 30437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:03.883138 30437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:03.883263 30437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:03.883339 30437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:03.919092 30437 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:45697
I20260928 16:42:03.919107 30564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:45697 every 8 connection(s)
I20260928 16:42:03.920835 30437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:42:03.921201 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 30437
I20260928 16:42:03.921383 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:42:03.922106 11231 external_mini_cluster.cc:1428] Setting key 7c2060d844c6df6654a2c0724a0f61ae
I20260928 16:42:03.939071 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.924212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:36061 (local address 127.10.247.254:43605)
0928 16:42:03.924826 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:03.924831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:03.926716 (+ 1885us) server_negotiation.cc:407] Connection header received
0928 16:42:03.929526 (+ 2810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:03.929531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:03.929644 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:03.929785 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:03.931695 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.933002 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.935657 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.936146 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.936275 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:03.938010 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:03.938018 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:03.938024 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:03.938052 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:03.938117 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:03.938132 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:03.938208 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:03.938532 (+ 324us) server_negotiation.cc:299] Negotiation successful
0928 16:42:03.938670 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":287,"threads_started":1}
I20260928 16:42:03.940635 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.925200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:43605 (local address 127.10.247.195:36061)
0928 16:42:03.926494 (+ 1294us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:03.926561 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:03.929075 (+ 2514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:03.930170 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:03.930186 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:03.930361 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:03.931473 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:03.931513 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.933216 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.933222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:03.935425 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:03.935450 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.936402 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.936408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:03.936763 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:03.937601 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:42:03.937604 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:03.937671 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:42:03.937786 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:03.938233 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:03.938247 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:03.938256 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 16:42:03.938503 (+ 247us) client_negotiation.cc:240] Negotiation successful
0928 16:42:03.939102 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":384,"threads_started":1}
I20260928 16:42:03.942391 30565 heartbeater.cc:347] Connected to a master server at 127.10.247.254:43605
I20260928 16:42:03.942790 30565 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:03.943771 30565 heartbeater.cc:511] Master 127.10.247.254:43605 requested a full tablet report, sending...
I20260928 16:42:03.945529 30116 ts_manager.cc:194] Registered new tserver with Master: 5f2f92d5ab9f4826ae86beca2802321d (127.10.247.195:45697)
I20260928 16:42:03.946508 30116 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.195:36061
I20260928 16:42:03.952416 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:42:03.964257 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.957480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52916 (local address 127.10.247.254:43605)
0928 16:42:03.957686 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:03.957691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:03.958218 (+ 527us) server_negotiation.cc:407] Connection header received
0928 16:42:03.958394 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:03.958398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:03.958482 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:03.958575 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:03.959271 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.960062 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.961671 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.962109 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.962208 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:03.963571 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:03.963575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:03.963579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:03.963610 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:03.963647 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:03.963660 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:03.963784 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:03.963912 (+ 128us) server_negotiation.cc:299] Negotiation successful
0928 16:42:03.963972 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:42:03.968953 30116 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:42:03.976723 30116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:42:04.003950 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.993119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:36973 (local address 127.0.0.1:40584)
0928 16:42:03.994343 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:03.994406 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:03.994669 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:03.996169 (+ 1500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:03.996178 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:03.996216 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:03.996843 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:03.996862 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.998875 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:03.998885 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.001234 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:04.001251 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.002814 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.002821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.003061 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.003122 (+ 61us) client_negotiation.cc:769] Sending connection context
0928 16:42:04.003274 (+ 152us) client_negotiation.cc:240] Negotiation successful
0928 16:42:04.003524 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"thread_start_us":628,"threads_started":1}
I20260928 16:42:04.003968 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.993673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.10.247.193:36973)
0928 16:42:03.995350 (+ 1677us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:03.995358 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:03.995388 (+ 30us) server_negotiation.cc:407] Connection header received
0928 16:42:03.995536 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:03.995545 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:03.995876 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:03.997114 (+ 1238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:03.997428 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:03.998731 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.001457 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.002591 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.002814 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.003132 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:04.003360 (+ 228us) server_negotiation.cc:299] Negotiation successful
0928 16:42:04.003577 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1490,"thread_start_us":1317,"threads_started":1}
I20260928 16:42:04.011417 30237 tablet_service.cc:1467] Processing CreateTablet for tablet 21ad3cf28b5e41f1a1a0ca445d265e6f (DEFAULT_TABLE table=test-table [id=05794adfc66648ae96698c8ba417a1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:04.014041 30237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ad3cf28b5e41f1a1a0ca445d265e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:04.019018 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.994865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:45697 (local address 127.0.0.1:37928)
0928 16:42:04.006338 (+ 11473us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:04.006363 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:04.006541 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:04.007916 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:04.007924 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:04.007962 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:04.008422 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:04.008436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.010917 (+ 2481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.010924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.013489 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:04.013520 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.018116 (+ 4596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.018122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.018275 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.018307 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 16:42:04.018405 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 16:42:04.018592 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11282,"thread_start_us":9289,"threads_started":1}
I20260928 16:42:04.019989 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.995581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.10.247.195:45697)
0928 16:42:03.997707 (+ 2126us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:03.997717 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:04.006712 (+ 8995us) server_negotiation.cc:407] Connection header received
0928 16:42:04.006883 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:04.006895 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:04.007265 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:04.007511 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:04.008705 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.010674 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.015672 (+ 4998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.017449 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.017722 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.017946 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:04.018512 (+ 566us) server_negotiation.cc:299] Negotiation successful
0928 16:42:04.018730 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1826,"thread_start_us":1642,"threads_started":1}
I20260928 16:42:04.023541 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.993856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:33085 (local address 127.0.0.1:44452)
0928 16:42:04.004304 (+ 10448us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:04.004327 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:04.004455 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:04.015456 (+ 11001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:04.015463 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:04.015506 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:04.015757 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:04.015770 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.017764 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.017771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.019790 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:04.019807 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.022634 (+ 2827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.022641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.022869 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.022907 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 16:42:04.023015 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 16:42:04.023209 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10300,"thread_start_us":20079,"threads_started":1}
I20260928 16:42:04.023663 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:03.994435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44452 (local address 127.10.247.194:33085)
0928 16:42:04.014662 (+ 20227us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:04.014672 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:04.014712 (+ 40us) server_negotiation.cc:407] Connection header received
0928 16:42:04.014824 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:04.014831 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:04.015186 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:04.015402 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:04.015930 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.017552 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.020004 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.022396 (+ 2392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.022597 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.022883 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:04.023085 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 16:42:04.023335 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20025,"thread_start_us":19909,"threads_started":1}
I20260928 16:42:04.025538 30500 tablet_service.cc:1467] Processing CreateTablet for tablet 21ad3cf28b5e41f1a1a0ca445d265e6f (DEFAULT_TABLE table=test-table [id=05794adfc66648ae96698c8ba417a1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:04.028041 30500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ad3cf28b5e41f1a1a0ca445d265e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:04.027925 30368 tablet_service.cc:1467] Processing CreateTablet for tablet 21ad3cf28b5e41f1a1a0ca445d265e6f (DEFAULT_TABLE table=test-table [id=05794adfc66648ae96698c8ba417a1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:04.029223 30368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ad3cf28b5e41f1a1a0ca445d265e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:04.032809 30584 tablet_bootstrap.cc:492] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: Bootstrap starting.
I20260928 16:42:04.038024 30584 tablet_bootstrap.cc:654] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:04.039772 30584 log.cc:824] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:04.042876 30587 tablet_bootstrap.cc:492] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: Bootstrap starting.
I20260928 16:42:04.043833 30584 tablet_bootstrap.cc:492] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: No bootstrap required, opened a new log
I20260928 16:42:04.044246 30584 ts_tablet_manager.cc:1397] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260928 16:42:04.048098 30587 tablet_bootstrap.cc:654] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:04.049253 30588 tablet_bootstrap.cc:492] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: Bootstrap starting.
I20260928 16:42:04.049655 30587 log.cc:824] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:04.053552 30588 tablet_bootstrap.cc:654] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:04.054417 30587 tablet_bootstrap.cc:492] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: No bootstrap required, opened a new log
I20260928 16:42:04.054797 30587 ts_tablet_manager.cc:1397] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260928 16:42:04.055066 30588 log.cc:824] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:04.056559 30584 raft_consensus.cc:348] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } }
I20260928 16:42:04.057215 30584 raft_consensus.cc:374] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:04.057331 30584 raft_consensus.cc:729] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b372e0244efd4128a3ab91e951985e11, State: Initialized, Role: FOLLOWER
I20260928 16:42:04.058228 30584 consensus_queue.cc:260] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } }
I20260928 16:42:04.060369 30584 ts_tablet_manager.cc:1428] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: Time spent starting tablet: real 0.016s user 0.014s sys 0.003s
I20260928 16:42:04.060830 30588 tablet_bootstrap.cc:492] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: No bootstrap required, opened a new log
I20260928 16:42:04.061172 30588 ts_tablet_manager.cc:1397] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260928 16:42:04.064992 30587 raft_consensus.cc:348] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } }
I20260928 16:42:04.065595 30587 raft_consensus.cc:374] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:04.065708 30587 raft_consensus.cc:729] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17c264ec094b4b17892e518159b0137d, State: Initialized, Role: FOLLOWER
I20260928 16:42:04.066443 30587 consensus_queue.cc:260] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } }
I20260928 16:42:04.067848 30433 heartbeater.cc:503] Master 127.10.247.254:43605 was elected leader, sending a full tablet report...
I20260928 16:42:04.068833 30587 ts_tablet_manager.cc:1428] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
W20260928 16:42:04.070111 30303 tablet.cc:2369] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:04.071831 30588 raft_consensus.cc:348] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } }
I20260928 16:42:04.072317 30588 raft_consensus.cc:374] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:04.072441 30588 raft_consensus.cc:729] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2f92d5ab9f4826ae86beca2802321d, State: Initialized, Role: FOLLOWER
I20260928 16:42:04.073241 30588 consensus_queue.cc:260] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } }
I20260928 16:42:04.074575 30565 heartbeater.cc:503] Master 127.10.247.254:43605 was elected leader, sending a full tablet report...
I20260928 16:42:04.075273 30588 ts_tablet_manager.cc:1428] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260928 16:42:04.096882 30591 raft_consensus.cc:482] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:42:04.097354 30591 raft_consensus.cc:504] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } }
I20260928 16:42:04.099970 30591 leader_election.cc:288] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f2f92d5ab9f4826ae86beca2802321d (127.10.247.195:45697), 17c264ec094b4b17892e518159b0137d (127.10.247.194:33085)
I20260928 16:42:04.109131 30595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:04.100522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:33085 (local address 127.10.247.193:49473)
0928 16:42:04.101680 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:04.101694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:04.101835 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:04.102273 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:04.102277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:04.102297 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:04.102771 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:04.102784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.104149 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.104154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.106771 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:04.106788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.108192 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.108199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.108376 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.108418 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 16:42:04.108530 (+ 112us) client_negotiation.cc:240] Negotiation successful
0928 16:42:04.108715 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":105,"thread_start_us":830,"threads_started":1}
I20260928 16:42:04.109099 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:04.100697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:49473 (local address 127.10.247.194:33085)
0928 16:42:04.101204 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:04.101210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:04.101797 (+ 587us) server_negotiation.cc:407] Connection header received
0928 16:42:04.101971 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:04.101977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:04.102140 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:04.102245 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:04.102968 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.104021 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.107013 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.107983 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.108166 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.108249 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:04.108584 (+ 335us) server_negotiation.cc:299] Negotiation successful
0928 16:42:04.108793 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357}
I20260928 16:42:04.109237 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:04.100551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:58107 (local address 127.10.247.195:45697)
0928 16:42:04.101018 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:04.101026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:04.101570 (+ 544us) server_negotiation.cc:407] Connection header received
0928 16:42:04.101824 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:04.101829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:04.101921 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:04.102001 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:04.102701 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.103812 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.107062 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.108143 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.108308 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.108384 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:04.108823 (+ 439us) server_negotiation.cc:299] Negotiation successful
0928 16:42:04.108987 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260928 16:42:04.109942 30594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:04.100306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:45697 (local address 127.10.247.193:58107)
0928 16:42:04.101395 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:04.101421 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:04.101652 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:04.102039 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:04.102044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:04.102069 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:04.102547 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:04.102561 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.103946 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.103949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.106771 (+ 2822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:04.106788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.108424 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.108430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:04.108633 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.108668 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 16:42:04.109484 (+ 816us) client_negotiation.cc:240] Negotiation successful
0928 16:42:04.109651 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":130,"thread_start_us":599,"threads_started":1}
I20260928 16:42:04.110469 30388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ad3cf28b5e41f1a1a0ca445d265e6f" candidate_uuid: "b372e0244efd4128a3ab91e951985e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17c264ec094b4b17892e518159b0137d" is_pre_election: true
I20260928 16:42:04.110951 30520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ad3cf28b5e41f1a1a0ca445d265e6f" candidate_uuid: "b372e0244efd4128a3ab91e951985e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2f92d5ab9f4826ae86beca2802321d" is_pre_election: true
I20260928 16:42:04.111121 30388 raft_consensus.cc:2454] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b372e0244efd4128a3ab91e951985e11 in term 0.
I20260928 16:42:04.111495 30520 raft_consensus.cc:2454] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b372e0244efd4128a3ab91e951985e11 in term 0.
I20260928 16:42:04.111936 30191 leader_election.cc:302] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17c264ec094b4b17892e518159b0137d, b372e0244efd4128a3ab91e951985e11; no voters:
I20260928 16:42:04.112416 30591 raft_consensus.cc:2781] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:42:04.112578 30591 raft_consensus.cc:482] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:42:04.112709 30591 raft_consensus.cc:3037] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 0 FOLLOWER]: Advancing to term 1
W20260928 16:42:04.114205 30434 tablet.cc:2369] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:04.115968 30591 raft_consensus.cc:504] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } }
I20260928 16:42:04.116794 30591 leader_election.cc:288] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [CANDIDATE]: Term 1 election: Requested vote from peers 5f2f92d5ab9f4826ae86beca2802321d (127.10.247.195:45697), 17c264ec094b4b17892e518159b0137d (127.10.247.194:33085)
I20260928 16:42:04.117516 30388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ad3cf28b5e41f1a1a0ca445d265e6f" candidate_uuid: "b372e0244efd4128a3ab91e951985e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17c264ec094b4b17892e518159b0137d"
I20260928 16:42:04.117532 30520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ad3cf28b5e41f1a1a0ca445d265e6f" candidate_uuid: "b372e0244efd4128a3ab91e951985e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2f92d5ab9f4826ae86beca2802321d"
I20260928 16:42:04.117759 30388 raft_consensus.cc:3037] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:04.117770 30520 raft_consensus.cc:3037] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:04.120343 30520 raft_consensus.cc:2454] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b372e0244efd4128a3ab91e951985e11 in term 1.
I20260928 16:42:04.120368 30388 raft_consensus.cc:2454] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b372e0244efd4128a3ab91e951985e11 in term 1.
I20260928 16:42:04.121011 30190 leader_election.cc:302] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 17c264ec094b4b17892e518159b0137d, 5f2f92d5ab9f4826ae86beca2802321d, b372e0244efd4128a3ab91e951985e11; no voters:
I20260928 16:42:04.121429 30591 raft_consensus.cc:2781] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:04.122004 30591 raft_consensus.cc:686] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 1 LEADER]: Becoming Leader. State: Replica: b372e0244efd4128a3ab91e951985e11, State: Running, Role: LEADER
I20260928 16:42:04.122684 30591 consensus_queue.cc:237] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } }
I20260928 16:42:04.128938 30115 catalog_manager.cc:5495] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 reported cstate change: term changed from 0 to 1, leader changed from <none> to b372e0244efd4128a3ab91e951985e11 (127.10.247.193). New cstate: current_term: 1 leader_uuid: "b372e0244efd4128a3ab91e951985e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b372e0244efd4128a3ab91e951985e11" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 36973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 } health_report { overall_health: UNKNOWN } } }
W20260928 16:42:04.173056 30566 tablet.cc:2369] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:04.179353 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:04.172489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40596 (local address 127.10.247.193:36973)
0928 16:42:04.172762 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:04.172770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:04.172900 (+ 130us) server_negotiation.cc:407] Connection header received
0928 16:42:04.173164 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:04.173170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:04.173326 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:04.173519 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:04.173956 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.174895 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.176509 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.177123 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.177244 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.177836 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:04.178580 (+ 744us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:04.178736 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:04.179032 (+ 296us) server_negotiation.cc:299] Negotiation successful
0928 16:42:04.179097 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:42:04.190644 30388 raft_consensus.cc:1261] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [term 1 FOLLOWER]: Refusing update from remote peer b372e0244efd4128a3ab91e951985e11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:04.190644 30520 raft_consensus.cc:1261] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [term 1 FOLLOWER]: Refusing update from remote peer b372e0244efd4128a3ab91e951985e11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:04.192107 30596 consensus_queue.cc:1035] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2f92d5ab9f4826ae86beca2802321d" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 45697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:04.192674 30591 consensus_queue.cc:1035] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17c264ec094b4b17892e518159b0137d" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 33085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:04.206789 30599 mvcc.cc:205] Tried to move back new op lower bound from 7334353814271594496 to 7334353814011777024. Current Snapshot: MvccSnapshot[applied={T|T < 7334353814271594496}]
I20260928 16:42:04.223935 30116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:42:04.224294 30116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:42:04.228871 30116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cf3e19be6694486973750cc87b443ec: Sending DeleteTablet for 3 replicas of tablet 21ad3cf28b5e41f1a1a0ca445d265e6f
I20260928 16:42:04.230444 30368 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ad3cf28b5e41f1a1a0ca445d265e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:04 UTC) from {username='slave'} at 127.0.0.1:44452
I20260928 16:42:04.230845 30237 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ad3cf28b5e41f1a1a0ca445d265e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:04 UTC) from {username='slave'} at 127.0.0.1:40584
I20260928 16:42:04.231714 30500 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ad3cf28b5e41f1a1a0ca445d265e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:04 UTC) from {username='slave'} at 127.0.0.1:37928
I20260928 16:42:04.233387 30611 tablet_replica.cc:330] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: stopping tablet replica
I20260928 16:42:04.233831 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 30175
I20260928 16:42:04.234048 30611 raft_consensus.cc:2229] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:42:04.234788 30611 raft_consensus.cc:2258] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:04.235803 30610 tablet_replica.cc:330] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: stopping tablet replica
I20260928 16:42:04.236328 30610 raft_consensus.cc:2229] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:42:04.237222 30610 raft_consensus.cc:2258] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:04.238121 30609 tablet_replica.cc:330] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: stopping tablet replica
I20260928 16:42:04.238606 30609 raft_consensus.cc:2229] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:42:04.239184 30609 raft_consensus.cc:2258] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:04.241498 30610 ts_tablet_manager.cc:1905] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:04.241492 30611 ts_tablet_manager.cc:1905] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:04.248951 30609 ts_tablet_manager.cc:1905] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:04.249786 30611 ts_tablet_manager.cc:1918] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:04.249874 30610 ts_tablet_manager.cc:1918] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:04.250128 30611 log.cc:1196] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/21ad3cf28b5e41f1a1a0ca445d265e6f
I20260928 16:42:04.250108 30610 log.cc:1196] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/21ad3cf28b5e41f1a1a0ca445d265e6f
I20260928 16:42:04.250850 30611 ts_tablet_manager.cc:1939] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 5f2f92d5ab9f4826ae86beca2802321d: Deleting consensus metadata
I20260928 16:42:04.250859 30610 ts_tablet_manager.cc:1939] T 21ad3cf28b5e41f1a1a0ca445d265e6f P b372e0244efd4128a3ab91e951985e11: Deleting consensus metadata
I20260928 16:42:04.253451 30100 catalog_manager.cc:4841] TS 5f2f92d5ab9f4826ae86beca2802321d (127.10.247.195:45697): tablet 21ad3cf28b5e41f1a1a0ca445d265e6f (table test-table [id=05794adfc66648ae96698c8ba417a1ac]) successfully deleted
I20260928 16:42:04.253904 30100 catalog_manager.cc:4841] TS b372e0244efd4128a3ab91e951985e11 (127.10.247.193:36973): tablet 21ad3cf28b5e41f1a1a0ca445d265e6f (table test-table [id=05794adfc66648ae96698c8ba417a1ac]) successfully deleted
I20260928 16:42:04.255714 30609 ts_tablet_manager.cc:1918] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:04.255891 30609 log.cc:1196] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/21ad3cf28b5e41f1a1a0ca445d265e6f
I20260928 16:42:04.256496 30609 ts_tablet_manager.cc:1939] T 21ad3cf28b5e41f1a1a0ca445d265e6f P 17c264ec094b4b17892e518159b0137d: Deleting consensus metadata
I20260928 16:42:04.258391 30101 catalog_manager.cc:4841] TS 17c264ec094b4b17892e518159b0137d (127.10.247.194:33085): tablet 21ad3cf28b5e41f1a1a0ca445d265e6f (table test-table [id=05794adfc66648ae96698c8ba417a1ac]) successfully deleted
I20260928 16:42:04.258810 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:04.236754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40602 (local address 127.10.247.193:36973)
0928 16:42:04.238636 (+ 1882us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:04.238644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:04.240069 (+ 1425us) server_negotiation.cc:407] Connection header received
0928 16:42:04.240893 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:04.240901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:04.241034 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:04.241124 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:04.241865 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.243161 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.245197 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.245851 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.245979 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.256588 (+ 10609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:04.256670 (+ 82us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:04.256695 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:04.257742 (+ 1047us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:04.257853 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:04.257890 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:04.258036 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:04.258287 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 16:42:04.258447 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1741}
I20260928 16:42:04.260162 30297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:04.370517 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 30175
I20260928 16:42:04.390885 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 30306
I20260928 16:42:04.405391 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:04.391967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44462 (local address 127.10.247.194:33085)
0928 16:42:04.392170 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:04.392175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:04.392196 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:42:04.392358 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:04.392363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:04.392485 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:04.392580 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:04.393141 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.393999 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.395407 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.395864 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.395978 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.403439 (+ 7461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:04.403454 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:04.403473 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:04.404281 (+ 808us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:04.404353 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:04.404384 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:04.404498 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:04.404867 (+ 369us) server_negotiation.cc:299] Negotiation successful
0928 16:42:04.405011 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 16:42:04.406199 30428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:04.502864 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 30306
I20260928 16:42:04.520009 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 30437
I20260928 16:42:04.528424 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:04.521077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37934 (local address 127.10.247.195:45697)
0928 16:42:04.521275 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:04.521281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:04.521340 (+ 59us) server_negotiation.cc:407] Connection header received
0928 16:42:04.521593 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:04.521598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:04.521746 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:04.522023 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:04.522551 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.523411 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.524968 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:04.525444 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:04.525555 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:04.526805 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:04.526817 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:04.526837 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:04.527621 (+ 784us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:04.527687 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:04.527711 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:04.527813 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:04.527966 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 16:42:04.528089 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:42:04.529268 30560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:04.625026 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 30437
I20260928 16:42:04.642290 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 30085
I20260928 16:42:04.643743 30146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:04.768723 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 30085
2026-09-28T16:42:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3251 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:04 dist-test-slave-9fn6 krb5kdc[30619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:04 dist-test-slave-9fn6 krb5kdc[30619](info): set up 2 sockets
Sep 28 16:42:04 dist-test-slave-9fn6 krb5kdc[30619](info): commencing operation
krb5kdc: starting...
W20260928 16:42:05.825410 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:06 dist-test-slave-9fn6 krb5kdc[30619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:42:06.152644 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37305
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:06.560325 30632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:06.560750 30632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:06.560897 30632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:06.568820 30632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:06.568961 30632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:06.569046 30632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:06.569131 30632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:06.569195 30632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:06.580418 30632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:06.582255 30632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:06.583890 30632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:06.597262 30637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:06.597262 30638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:06.597613 30640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:06.597901 30632 server_base.cc:1034] running on GCE node
I20260928 16:42:06.598995 30632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:06.600857 30632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:06.602075 30632 hybrid_clock.cc:648] HybridClock initialized: now 1790613726602019 us; error 53 us; skew 500 ppm
I20260928 16:42:06.602473 30632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:06 dist-test-slave-9fn6 krb5kdc[30619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613726, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:06.605948 30632 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:42:06.607705 30632 webserver.cc:460] Webserver started at http://127.10.247.254:36707/ using document root <none> and password file <none>
I20260928 16:42:06.608541 30632 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:06.608705 30632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:06.609074 30632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:06.612782 30632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "3750963bd2b944dbb03b3216601981bb"
format_stamp: "Formatted at 2026-09-28 16:42:06 on dist-test-slave-9fn6"
server_key: "925492c83f5ea6c80dc36eed4aea390f"
server_key_iv: "c63218b7e805fac13a0a796231be21c3"
server_key_version: "encryptionkey@0"
I20260928 16:42:06.613740 30632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "3750963bd2b944dbb03b3216601981bb"
format_stamp: "Formatted at 2026-09-28 16:42:06 on dist-test-slave-9fn6"
server_key: "925492c83f5ea6c80dc36eed4aea390f"
server_key_iv: "c63218b7e805fac13a0a796231be21c3"
server_key_version: "encryptionkey@0"
I20260928 16:42:06.620100 30632 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260928 16:42:06.624328 30647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:06.626091 30632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 16:42:06.626288 30632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "3750963bd2b944dbb03b3216601981bb"
format_stamp: "Formatted at 2026-09-28 16:42:06 on dist-test-slave-9fn6"
server_key: "925492c83f5ea6c80dc36eed4aea390f"
server_key_iv: "c63218b7e805fac13a0a796231be21c3"
server_key_version: "encryptionkey@0"
I20260928 16:42:06.626528 30632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:06.639454 30632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:06.644902 30632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:06.645246 30632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:06.665889 30632 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:37305
I20260928 16:42:06.665920 30698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:37305 every 8 connection(s)
I20260928 16:42:06.667613 30632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:06.672335 30699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:06.678004 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 30632
I20260928 16:42:06.678242 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:06.678902 11231 external_mini_cluster.cc:1428] Setting key b87eb8e215748ce227e944c760c01325
I20260928 16:42:06.685732 30699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb: Bootstrap starting.
I20260928 16:42:06.689508 30699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:06.690737 30699 log.cc:824] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:06.694288 30699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb: No bootstrap required, opened a new log
Sep 28 16:42:06 dist-test-slave-9fn6 krb5kdc[30619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:06.703295 30699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3750963bd2b944dbb03b3216601981bb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37305 } }
I20260928 16:42:06.703763 30699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:06.703888 30699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3750963bd2b944dbb03b3216601981bb, State: Initialized, Role: FOLLOWER
I20260928 16:42:06.704469 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:06.681316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.10.247.254:37305)
0928 16:42:06.682302 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:06.682317 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:06.682370 (+ 53us) server_negotiation.cc:407] Connection header received
0928 16:42:06.684329 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:06.684358 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:06.684893 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:06.685344 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:06.687065 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:06.688591 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:06.690225 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:06.691130 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:06.691339 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:06.695878 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:06.695888 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:06.695907 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:06.695955 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:06.699476 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:06.700326 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:06.700334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:06.700343 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:06.700559 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:06.701009 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:06.701013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:06.701015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:06.701757 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:06.702092 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:06.702589 (+ 497us) server_negotiation.cc:299] Negotiation successful
0928 16:42:06.703135 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":430,"threads_started":1}
I20260928 16:42:06.704636 30699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3750963bd2b944dbb03b3216601981bb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37305 } }
I20260928 16:42:06.704990 30699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:06.705156 30699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:06.705363 30699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:06.708374 30699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3750963bd2b944dbb03b3216601981bb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37305 } }
I20260928 16:42:06.709014 30699 leader_election.cc:302] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3750963bd2b944dbb03b3216601981bb; no voters:
I20260928 16:42:06.709671 30699 leader_election.cc:288] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:06.709802 30704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:06.710587 30704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [term 1 LEADER]: Becoming Leader. State: Replica: 3750963bd2b944dbb03b3216601981bb, State: Running, Role: LEADER
I20260928 16:42:06.711229 30704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3750963bd2b944dbb03b3216601981bb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37305 } }
I20260928 16:42:06.712373 30699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:06.714567 30705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3750963bd2b944dbb03b3216601981bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3750963bd2b944dbb03b3216601981bb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37305 } } }
I20260928 16:42:06.715049 30705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:06.716943 30711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:06.716773 30706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3750963bd2b944dbb03b3216601981bb. Latest consensus state: current_term: 1 leader_uuid: "3750963bd2b944dbb03b3216601981bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3750963bd2b944dbb03b3216601981bb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37305 } } }
I20260928 16:42:06.717141 30706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:06.723452 30711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:06.735052 30711 catalog_manager.cc:1347] Generated new cluster ID: e00a337d23204579b3b231794bd4dffe
I20260928 16:42:06.735219 30711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:06.748317 30711 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:06.749982 30711 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:06.765067 30711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb: Generated new TSK 0
I20260928 16:42:06.766734 30711 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.247.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
Entry for principal kudu/127.10.247.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab.
I20260928 16:42:06.850133 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37305
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:07.256570 30725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:07.257061 30725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:07.257215 30725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:07.265669 30725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:07.265827 30725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:07.265996 30725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:42:07.277504 30725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.193.keytab
--principal=kudu/127.10.247.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:07.279357 30725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:07.281129 30725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:07.297546 30731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:07.297499 30730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:07.297582 30733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:07.298561 30725 server_base.cc:1034] running on GCE node
I20260928 16:42:07.299387 30725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:07.300825 30725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:07.302088 30725 hybrid_clock.cc:648] HybridClock initialized: now 1790613727302026 us; error 60 us; skew 500 ppm
I20260928 16:42:07.302518 30725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:07 dist-test-slave-9fn6 krb5kdc[30619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613727, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:07.306435 30725 init.cc:377] Logged in from keytab as kudu/127.10.247.193@KRBTEST.COM (short username kudu)
I20260928 16:42:07.308391 30725 webserver.cc:460] Webserver started at http://127.10.247.193:33581/ using document root <none> and password file <none>
I20260928 16:42:07.309338 30725 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:07.309477 30725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:07.309846 30725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:07.313767 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "4e213c59a73a4213b0e79994f84ec709"
format_stamp: "Formatted at 2026-09-28 16:42:07 on dist-test-slave-9fn6"
server_key: "5be367d3e53ada1c817eb6fb11e771ee"
server_key_iv: "1d52170b8e782087e549d365fbdfe9d2"
server_key_version: "encryptionkey@0"
I20260928 16:42:07.314720 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "4e213c59a73a4213b0e79994f84ec709"
format_stamp: "Formatted at 2026-09-28 16:42:07 on dist-test-slave-9fn6"
server_key: "5be367d3e53ada1c817eb6fb11e771ee"
server_key_iv: "1d52170b8e782087e549d365fbdfe9d2"
server_key_version: "encryptionkey@0"
I20260928 16:42:07.321627 30725 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:42:07.326301 30740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:07.328137 30725 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:42:07.328362 30725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "4e213c59a73a4213b0e79994f84ec709"
format_stamp: "Formatted at 2026-09-28 16:42:07 on dist-test-slave-9fn6"
server_key: "5be367d3e53ada1c817eb6fb11e771ee"
server_key_iv: "1d52170b8e782087e549d365fbdfe9d2"
server_key_version: "encryptionkey@0"
I20260928 16:42:07.328678 30725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:07.387389 30725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:07.393110 30725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:07.393450 30725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:07.394716 30725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:07.396528 30725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:07.396646 30725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:07.396799 30725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:07.396855 30725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:07.432473 30725 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:45495
I20260928 16:42:07.432525 30852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:45495 every 8 connection(s)
I20260928 16:42:07.434156 30725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:42:07.435461 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 30725
I20260928 16:42:07.435694 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:42:07.436347 11231 external_mini_cluster.cc:1428] Setting key 71c94df9cf10f036ab549cd13bcd5bc4
Sep 28 16:42:07 dist-test-slave-9fn6 krb5kdc[30619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613727, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.193@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:07.459205 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:07.437611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:39925 (local address 127.10.247.254:37305)
0928 16:42:07.438857 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:07.438865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:07.439700 (+ 835us) server_negotiation.cc:407] Connection header received
0928 16:42:07.441939 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:07.441947 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:07.442043 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:07.442165 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:07.444014 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:07.445190 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:07.447075 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:07.447705 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:07.447850 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:07.452263 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:07.452269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:07.452275 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:07.452328 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:07.455614 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:07.456474 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:07.456479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:07.456482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:07.456679 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:07.457117 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:07.457122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:07.457125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:07.457503 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:07.457655 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:07.458481 (+ 826us) server_negotiation.cc:299] Negotiation successful
0928 16:42:07.458761 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1089,"thread_start_us":954,"threads_started":1}
I20260928 16:42:07.460420 30856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:07.438529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37305 (local address 127.10.247.193:39925)
0928 16:42:07.439490 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:07.439546 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:07.441613 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:07.442440 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:07.442450 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:07.443101 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:07.443819 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:07.443842 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:07.445357 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:07.445361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:07.446828 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:07.446843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:07.447944 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:07.447948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:07.448212 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:07.449023 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:07.449051 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:07.451926 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:07.455786 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:07.455796 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:07.455816 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:07.456351 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:07.456790 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:07.456794 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:07.456796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:07.457026 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:07.457621 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:07.457628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:07.458079 (+ 451us) client_negotiation.cc:769] Sending connection context
0928 16:42:07.458357 (+ 278us) client_negotiation.cc:240] Negotiation successful
0928 16:42:07.458928 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":349,"threads_started":1}
I20260928 16:42:07.462529 30853 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37305
I20260928 16:42:07.462935 30853 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:07.463898 30853 heartbeater.cc:511] Master 127.10.247.254:37305 requested a full tablet report, sending...
I20260928 16:42:07.466601 30664 ts_manager.cc:194] Registered new tserver with Master: 4e213c59a73a4213b0e79994f84ec709 (127.10.247.193:45495)
I20260928 16:42:07.468467 30664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.193@KRBTEST.COM'} at 127.10.247.193:39925
WARNING: no policy specified for kudu/127.10.247.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
Entry for principal kudu/127.10.247.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab.
I20260928 16:42:07.514616 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37305
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:07.912388 30859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:07.912879 30859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:07.913040 30859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:07.921986 30859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:07.922170 30859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:07.922304 30859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:42:07.936041 30859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.194.keytab
--principal=kudu/127.10.247.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:07.938001 30859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:07.939636 30859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:07.955380 30865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:07.955428 30864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:07.955454 30867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:07.956398 30859 server_base.cc:1034] running on GCE node
I20260928 16:42:07.957261 30859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:07.958473 30859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:07.959702 30859 hybrid_clock.cc:648] HybridClock initialized: now 1790613727959618 us; error 71 us; skew 500 ppm
I20260928 16:42:07.960075 30859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:07 dist-test-slave-9fn6 krb5kdc[30619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613727, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:07.963732 30859 init.cc:377] Logged in from keytab as kudu/127.10.247.194@KRBTEST.COM (short username kudu)
I20260928 16:42:07.965621 30859 webserver.cc:460] Webserver started at http://127.10.247.194:38005/ using document root <none> and password file <none>
I20260928 16:42:07.966506 30859 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:07.966641 30859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:07.967007 30859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:07.970727 30859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "401014e9a55746ff96804deed3897cb1"
format_stamp: "Formatted at 2026-09-28 16:42:07 on dist-test-slave-9fn6"
server_key: "8fcff6053170b5e3ce2949f412560ff8"
server_key_iv: "63f4fa19687c2ee571df0491ad8627d4"
server_key_version: "encryptionkey@0"
I20260928 16:42:07.971728 30859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "401014e9a55746ff96804deed3897cb1"
format_stamp: "Formatted at 2026-09-28 16:42:07 on dist-test-slave-9fn6"
server_key: "8fcff6053170b5e3ce2949f412560ff8"
server_key_iv: "63f4fa19687c2ee571df0491ad8627d4"
server_key_version: "encryptionkey@0"
I20260928 16:42:07.978543 30859 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260928 16:42:07.983059 30874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:07.985075 30859 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:42:07.985299 30859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "401014e9a55746ff96804deed3897cb1"
format_stamp: "Formatted at 2026-09-28 16:42:07 on dist-test-slave-9fn6"
server_key: "8fcff6053170b5e3ce2949f412560ff8"
server_key_iv: "63f4fa19687c2ee571df0491ad8627d4"
server_key_version: "encryptionkey@0"
I20260928 16:42:07.985549 30859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:08.014528 30859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:08.020269 30859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:08.020596 30859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:08.021896 30859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:08.023777 30859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:08.023860 30859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:08.023962 30859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:08.024006 30859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:08.058120 30859 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:34573
I20260928 16:42:08.058185 30986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:34573 every 8 connection(s)
I20260928 16:42:08.059717 30859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:42:08.060773 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 30859
I20260928 16:42:08.060994 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:42:08.061688 11231 external_mini_cluster.cc:1428] Setting key a5e5dc2f1b5a9fc9e40363de387c25d2
Sep 28 16:42:08 dist-test-slave-9fn6 krb5kdc[30619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613727, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.194@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:08.085836 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.063179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:38343 (local address 127.10.247.254:37305)
0928 16:42:08.063680 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:08.063686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:08.065020 (+ 1334us) server_negotiation.cc:407] Connection header received
0928 16:42:08.067908 (+ 2888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:08.067912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:08.067995 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:08.068100 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:08.070339 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.071193 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.073513 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.074017 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.074124 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.078920 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:08.078926 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:08.078930 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:08.078976 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:08.082044 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:08.082948 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:08.082953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:08.082956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:08.083111 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:08.083589 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:08.083593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:08.083602 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:08.083961 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:08.084121 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:08.085094 (+ 973us) server_negotiation.cc:299] Negotiation successful
0928 16:42:08.085405 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":257,"threads_started":1}
I20260928 16:42:08.087244 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.063854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37305 (local address 127.10.247.194:38343)
0928 16:42:08.064821 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:08.064887 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:08.067549 (+ 2662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:08.068394 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:08.068407 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:08.069266 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:08.070146 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.070171 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.071353 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.071359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.073320 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.073338 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.074269 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.074275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.074587 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.075478 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:08.075510 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:08.078581 (+ 3071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:08.082239 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:08.082249 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:08.082267 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:08.082792 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:08.083229 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:08.083235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:08.083239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:08.083462 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:08.084120 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:08.084130 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:08.084595 (+ 465us) client_negotiation.cc:769] Sending connection context
0928 16:42:08.084946 (+ 351us) client_negotiation.cc:240] Negotiation successful
0928 16:42:08.085655 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":334,"threads_started":1}
I20260928 16:42:08.089303 30987 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37305
I20260928 16:42:08.089735 30987 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:08.090868 30987 heartbeater.cc:511] Master 127.10.247.254:37305 requested a full tablet report, sending...
I20260928 16:42:08.093250 30664 ts_manager.cc:194] Registered new tserver with Master: 401014e9a55746ff96804deed3897cb1 (127.10.247.194:34573)
I20260928 16:42:08.094697 30664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.194@KRBTEST.COM'} at 127.10.247.194:38343
WARNING: no policy specified for kudu/127.10.247.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
Entry for principal kudu/127.10.247.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab.
I20260928 16:42:08.136413 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37305
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:42:08.479991 30853 heartbeater.cc:503] Master 127.10.247.254:37305 was elected leader, sending a full tablet report...
W20260928 16:42:08.527468 30993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:08.527878 30993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:08.528000 30993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:08.536507 30993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:08.536747 30993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:08.536962 30993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:42:08.548215 30993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.195.keytab
--principal=kudu/127.10.247.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:37305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:08.550088 30993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:08.551843 30993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:08.566979 31001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:08.567142 30999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:08.567116 30998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:08.568595 30993 server_base.cc:1034] running on GCE node
I20260928 16:42:08.569481 30993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:08.570765 30993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:08.572000 30993 hybrid_clock.cc:648] HybridClock initialized: now 1790613728571934 us; error 58 us; skew 500 ppm
I20260928 16:42:08.572428 30993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:08 dist-test-slave-9fn6 krb5kdc[30619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613728, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:08.576191 30993 init.cc:377] Logged in from keytab as kudu/127.10.247.195@KRBTEST.COM (short username kudu)
I20260928 16:42:08.578089 30993 webserver.cc:460] Webserver started at http://127.10.247.195:39363/ using document root <none> and password file <none>
I20260928 16:42:08.578992 30993 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:08.579110 30993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:08.579486 30993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:08.583181 30993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "692a89e1f1624d9cb5206cd7e0b9eec6"
format_stamp: "Formatted at 2026-09-28 16:42:08 on dist-test-slave-9fn6"
server_key: "12be1dca9d989cfbe590d72c7bc8a23f"
server_key_iv: "1cad79fba9293152ab8a015f13411355"
server_key_version: "encryptionkey@0"
I20260928 16:42:08.584038 30993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "692a89e1f1624d9cb5206cd7e0b9eec6"
format_stamp: "Formatted at 2026-09-28 16:42:08 on dist-test-slave-9fn6"
server_key: "12be1dca9d989cfbe590d72c7bc8a23f"
server_key_iv: "1cad79fba9293152ab8a015f13411355"
server_key_version: "encryptionkey@0"
I20260928 16:42:08.590287 30993 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260928 16:42:08.594522 31008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:08.596323 30993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:42:08.596521 30993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "692a89e1f1624d9cb5206cd7e0b9eec6"
format_stamp: "Formatted at 2026-09-28 16:42:08 on dist-test-slave-9fn6"
server_key: "12be1dca9d989cfbe590d72c7bc8a23f"
server_key_iv: "1cad79fba9293152ab8a015f13411355"
server_key_version: "encryptionkey@0"
I20260928 16:42:08.596819 30993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:08.613706 30993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:08.618868 30993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:08.619192 30993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:08.620371 30993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:08.622021 30993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:08.622117 30993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:08.622253 30993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:08.622314 30993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:08.658836 30993 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:42025
I20260928 16:42:08.658891 31120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:42025 every 8 connection(s)
I20260928 16:42:08.660439 30993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:42:08.662503 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 30993
I20260928 16:42:08.662722 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:42:08.663354 11231 external_mini_cluster.cc:1428] Setting key 389437e0b7b2b6d1cfbafd0651e28815
Sep 28 16:42:08 dist-test-slave-9fn6 krb5kdc[30619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613728, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.195@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:08.683884 31123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.663862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:56137 (local address 127.10.247.254:37305)
0928 16:42:08.664469 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:08.664476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:08.665769 (+ 1293us) server_negotiation.cc:407] Connection header received
0928 16:42:08.667858 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:08.667863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:08.668016 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:08.668137 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:08.670121 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.671067 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.672848 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.673318 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.673424 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.677514 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:08.677520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:08.677524 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:08.677564 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:08.680339 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:08.681240 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:08.681244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:08.681247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:08.681391 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:08.681865 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:08.681868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:08.681870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:08.682176 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:08.682321 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:08.683313 (+ 992us) server_negotiation.cc:299] Negotiation successful
0928 16:42:08.683536 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":288,"threads_started":1}
I20260928 16:42:08.685346 31124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.664595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:37305 (local address 127.10.247.195:56137)
0928 16:42:08.665597 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:08.665651 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:08.667556 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:08.668449 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:08.668459 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:08.669181 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:08.669932 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.669953 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.671232 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.671236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.672704 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.672720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.673569 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.673576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.673904 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.674672 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:08.674700 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:08.677266 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:08.680512 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:08.680522 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:08.680539 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:08.681124 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:08.681535 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:08.681538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:08.681541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:08.681771 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:08.682339 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:08.682346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:08.682817 (+ 471us) client_negotiation.cc:769] Sending connection context
0928 16:42:08.683172 (+ 355us) client_negotiation.cc:240] Negotiation successful
0928 16:42:08.683714 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":359,"threads_started":1}
I20260928 16:42:08.687009 31121 heartbeater.cc:347] Connected to a master server at 127.10.247.254:37305
I20260928 16:42:08.687399 31121 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:08.688275 31121 heartbeater.cc:511] Master 127.10.247.254:37305 requested a full tablet report, sending...
I20260928 16:42:08.690003 30664 ts_manager.cc:194] Registered new tserver with Master: 692a89e1f1624d9cb5206cd7e0b9eec6 (127.10.247.195:42025)
I20260928 16:42:08.690989 30664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.247.195@KRBTEST.COM'} at 127.10.247.195:56137
I20260928 16:42:08.700299 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:42:08.719981 31123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.705554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.10.247.254:37305)
0928 16:42:08.705795 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:08.705804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:08.706111 (+ 307us) server_negotiation.cc:407] Connection header received
0928 16:42:08.706328 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:08.706333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:08.706442 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:08.706537 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:08.708195 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.709053 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.710405 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.710895 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.711019 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.713888 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:08.713893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:08.713897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:08.713944 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:08.716838 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:08.717665 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:08.717671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:08.717674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:08.717848 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:08.718279 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:08.718282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:08.718284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:08.718704 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:08.718970 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:08.719350 (+ 380us) server_negotiation.cc:299] Negotiation successful
0928 16:42:08.719631 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:42:08.725095 30664 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:42:08.733680 30664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:42:08.761130 31135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.749514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:42025 (local address 127.0.0.1:41038)
0928 16:42:08.750522 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:08.750565 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:08.750775 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:08.752276 (+ 1501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:08.752287 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:08.752330 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:08.753060 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.753081 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.755169 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.755175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.758190 (+ 3015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.758205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.759921 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.759927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.760120 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.760172 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 16:42:08.760317 (+ 145us) client_negotiation.cc:240] Negotiation successful
0928 16:42:08.760571 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":144,"thread_start_us":1657,"threads_started":1}
I20260928 16:42:08.761772 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.750125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41038 (local address 127.10.247.195:42025)
0928 16:42:08.751503 (+ 1378us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:08.751514 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:08.751558 (+ 44us) server_negotiation.cc:407] Connection header received
0928 16:42:08.751690 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:08.751698 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:08.752012 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:08.752239 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:08.753320 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.754993 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.758390 (+ 3397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.759727 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.759921 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.760152 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:08.760778 (+ 626us) server_negotiation.cc:299] Negotiation successful
0928 16:42:08.760990 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1172,"thread_start_us":1006,"threads_started":1}
I20260928 16:42:08.762212 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.750745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60340 (local address 127.10.247.194:34573)
0928 16:42:08.751855 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:08.751864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:08.751889 (+ 25us) server_negotiation.cc:407] Connection header received
0928 16:42:08.751996 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:08.752003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:08.752234 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:08.753407 (+ 1173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:08.753660 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.755199 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.759217 (+ 4018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.760589 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.760778 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.760944 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:08.761721 (+ 777us) server_negotiation.cc:299] Negotiation successful
0928 16:42:08.761907 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":936,"threads_started":1}
I20260928 16:42:08.762441 31134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.749627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34573 (local address 127.0.0.1:60340)
0928 16:42:08.751210 (+ 1583us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:08.751227 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:08.751352 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:08.752469 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:08.752474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:08.752530 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:08.753169 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.753184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.755331 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.755340 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.759022 (+ 3682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.759036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.761105 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.761116 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.761322 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.761355 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:42:08.761440 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 16:42:08.761578 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1452,"mutex_wait_us":166,"spinlock_wait_cycles":6400,"thread_start_us":431,"threads_started":1}
I20260928 16:42:08.762709 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.752060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43926 (local address 127.10.247.193:45495)
0928 16:42:08.753217 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:08.753227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:08.753765 (+ 538us) server_negotiation.cc:407] Connection header received
0928 16:42:08.754125 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:08.754134 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:08.754377 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:08.754563 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:08.755462 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.756516 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.758431 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.759889 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.760089 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.760260 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:08.762035 (+ 1775us) server_negotiation.cc:299] Negotiation successful
0928 16:42:08.762221 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":942,"thread_start_us":947,"threads_started":1}
I20260928 16:42:08.763253 31139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:08.752809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:45495 (local address 127.0.0.1:43926)
0928 16:42:08.753649 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:08.753666 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:08.753888 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:08.754851 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:08.754855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:08.754880 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:08.755292 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.755309 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.756680 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.756684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.758280 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:08.758295 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:08.761396 (+ 3101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:08.761403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:08.761579 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:08.761617 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 16:42:08.761711 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 16:42:08.761853 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":3,"thread_start_us":564,"threads_started":1}
I20260928 16:42:08.766445 31056 tablet_service.cc:1467] Processing CreateTablet for tablet 26d629b6f8a340ac8a3e383236a743cf (DEFAULT_TABLE table=test-table [id=148b3d6f099a4456bae31656fcf5b3de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:08.766697 30788 tablet_service.cc:1467] Processing CreateTablet for tablet 26d629b6f8a340ac8a3e383236a743cf (DEFAULT_TABLE table=test-table [id=148b3d6f099a4456bae31656fcf5b3de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:08.766698 30922 tablet_service.cc:1467] Processing CreateTablet for tablet 26d629b6f8a340ac8a3e383236a743cf (DEFAULT_TABLE table=test-table [id=148b3d6f099a4456bae31656fcf5b3de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:08.768245 30922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d629b6f8a340ac8a3e383236a743cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:08.768252 30788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d629b6f8a340ac8a3e383236a743cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:08.768257 31056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d629b6f8a340ac8a3e383236a743cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:08.783473 31141 tablet_bootstrap.cc:492] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: Bootstrap starting.
I20260928 16:42:08.784391 31140 tablet_bootstrap.cc:492] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: Bootstrap starting.
I20260928 16:42:08.784488 31142 tablet_bootstrap.cc:492] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: Bootstrap starting.
I20260928 16:42:08.787643 31142 tablet_bootstrap.cc:654] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:08.787679 31141 tablet_bootstrap.cc:654] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:08.787945 31140 tablet_bootstrap.cc:654] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:08.788913 31142 log.cc:824] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:08.789024 31141 log.cc:824] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:08.789167 31140 log.cc:824] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:08.792685 31140 tablet_bootstrap.cc:492] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: No bootstrap required, opened a new log
I20260928 16:42:08.792685 31141 tablet_bootstrap.cc:492] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: No bootstrap required, opened a new log
I20260928 16:42:08.792682 31142 tablet_bootstrap.cc:492] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: No bootstrap required, opened a new log
I20260928 16:42:08.792997 31142 ts_tablet_manager.cc:1397] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260928 16:42:08.793023 31141 ts_tablet_manager.cc:1397] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260928 16:42:08.793040 31140 ts_tablet_manager.cc:1397] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260928 16:42:08.801065 31142 raft_consensus.cc:348] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } }
I20260928 16:42:08.801651 31142 raft_consensus.cc:374] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:08.801764 31142 raft_consensus.cc:729] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 401014e9a55746ff96804deed3897cb1, State: Initialized, Role: FOLLOWER
I20260928 16:42:08.802701 31142 consensus_queue.cc:260] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } }
I20260928 16:42:08.804270 31140 raft_consensus.cc:348] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } }
I20260928 16:42:08.804273 31141 raft_consensus.cc:348] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } }
I20260928 16:42:08.804841 31140 raft_consensus.cc:374] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:08.804842 31141 raft_consensus.cc:374] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:08.804962 31141 raft_consensus.cc:729] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e213c59a73a4213b0e79994f84ec709, State: Initialized, Role: FOLLOWER
I20260928 16:42:08.804952 31140 raft_consensus.cc:729] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 692a89e1f1624d9cb5206cd7e0b9eec6, State: Initialized, Role: FOLLOWER
I20260928 16:42:08.805853 31140 consensus_queue.cc:260] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } }
I20260928 16:42:08.805855 31141 consensus_queue.cc:260] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } }
I20260928 16:42:08.806057 30987 heartbeater.cc:503] Master 127.10.247.254:37305 was elected leader, sending a full tablet report...
I20260928 16:42:08.806917 31142 ts_tablet_manager.cc:1428] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260928 16:42:08.808271 31121 heartbeater.cc:503] Master 127.10.247.254:37305 was elected leader, sending a full tablet report...
I20260928 16:42:08.809140 31140 ts_tablet_manager.cc:1428] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260928 16:42:08.809151 31141 ts_tablet_manager.cc:1428] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260928 16:42:08.812254 30988 tablet.cc:2369] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:42:08.912410 31122 tablet.cc:2369] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:42:08.937265 30854 tablet.cc:2369] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:09.043864 31147 raft_consensus.cc:482] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:42:09.044286 31147 raft_consensus.cc:504] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } }
I20260928 16:42:09.046693 31147 leader_election.cc:288] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 692a89e1f1624d9cb5206cd7e0b9eec6 (127.10.247.195:42025), 401014e9a55746ff96804deed3897cb1 (127.10.247.194:34573)
I20260928 16:42:09.054652 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:09.047667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:59175 (local address 127.10.247.194:34573)
0928 16:42:09.047957 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:09.047964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:09.048664 (+ 700us) server_negotiation.cc:407] Connection header received
0928 16:42:09.048836 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:09.048840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:09.048915 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:09.048996 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:09.049657 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.050441 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.052683 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.053561 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.053736 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:09.053830 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:09.054219 (+ 389us) server_negotiation.cc:299] Negotiation successful
0928 16:42:09.054366 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260928 16:42:09.054633 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:09.047158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:46201 (local address 127.10.247.195:42025)
0928 16:42:09.047457 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:09.047476 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:09.048314 (+ 838us) server_negotiation.cc:407] Connection header received
0928 16:42:09.048569 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:09.048573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:09.048684 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:09.048771 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:09.049340 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.050170 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.052571 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.053480 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.053643 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:09.053742 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:09.054147 (+ 405us) server_negotiation.cc:299] Negotiation successful
0928 16:42:09.054318 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260928 16:42:09.054849 31150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:09.047299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:34573 (local address 127.10.247.193:59175)
0928 16:42:09.048487 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:09.048508 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:09.048686 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:09.049045 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:09.049050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:09.049076 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:09.049479 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:09.049488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.050595 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.050599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:09.052475 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:09.052492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.053838 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.053844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:09.054027 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:09.054052 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 16:42:09.054146 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 16:42:09.054335 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1063,"mutex_wait_us":100,"thread_start_us":859,"threads_started":1}
I20260928 16:42:09.054877 31149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:09.047013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:42025 (local address 127.10.247.193:46201)
0928 16:42:09.048134 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:09.048153 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:09.048398 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:09.048803 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:09.048808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:09.048835 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:09.049194 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:09.049209 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.050309 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.050313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:09.052359 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:09.052380 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.053758 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.053771 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:09.053958 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:09.054005 (+ 47us) client_negotiation.cc:769] Sending connection context
0928 16:42:09.054084 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 16:42:09.054278 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":27,"thread_start_us":734,"threads_started":1}
I20260928 16:42:09.056282 30942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d629b6f8a340ac8a3e383236a743cf" candidate_uuid: "4e213c59a73a4213b0e79994f84ec709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "401014e9a55746ff96804deed3897cb1" is_pre_election: true
I20260928 16:42:09.057106 30942 raft_consensus.cc:2454] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e213c59a73a4213b0e79994f84ec709 in term 0.
I20260928 16:42:09.057406 31076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d629b6f8a340ac8a3e383236a743cf" candidate_uuid: "4e213c59a73a4213b0e79994f84ec709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" is_pre_election: true
I20260928 16:42:09.058053 31076 raft_consensus.cc:2454] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e213c59a73a4213b0e79994f84ec709 in term 0.
I20260928 16:42:09.058133 30744 leader_election.cc:302] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 401014e9a55746ff96804deed3897cb1, 4e213c59a73a4213b0e79994f84ec709; no voters:
I20260928 16:42:09.058686 31147 raft_consensus.cc:2781] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:42:09.058841 31147 raft_consensus.cc:482] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:42:09.058946 31147 raft_consensus.cc:3037] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:09.061615 31147 raft_consensus.cc:504] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } }
I20260928 16:42:09.062367 31147 leader_election.cc:288] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [CANDIDATE]: Term 1 election: Requested vote from peers 692a89e1f1624d9cb5206cd7e0b9eec6 (127.10.247.195:42025), 401014e9a55746ff96804deed3897cb1 (127.10.247.194:34573)
I20260928 16:42:09.063042 30942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d629b6f8a340ac8a3e383236a743cf" candidate_uuid: "4e213c59a73a4213b0e79994f84ec709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "401014e9a55746ff96804deed3897cb1"
I20260928 16:42:09.063071 31076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d629b6f8a340ac8a3e383236a743cf" candidate_uuid: "4e213c59a73a4213b0e79994f84ec709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6"
I20260928 16:42:09.063349 30942 raft_consensus.cc:3037] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:09.063341 31076 raft_consensus.cc:3037] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:09.067044 31076 raft_consensus.cc:2454] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e213c59a73a4213b0e79994f84ec709 in term 1.
I20260928 16:42:09.067044 30942 raft_consensus.cc:2454] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e213c59a73a4213b0e79994f84ec709 in term 1.
I20260928 16:42:09.067674 30744 leader_election.cc:302] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 401014e9a55746ff96804deed3897cb1, 4e213c59a73a4213b0e79994f84ec709; no voters:
I20260928 16:42:09.068105 31147 raft_consensus.cc:2781] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:09.068717 31147 raft_consensus.cc:686] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 1 LEADER]: Becoming Leader. State: Replica: 4e213c59a73a4213b0e79994f84ec709, State: Running, Role: LEADER
I20260928 16:42:09.069425 31147 consensus_queue.cc:237] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } }
I20260928 16:42:09.075829 30664 catalog_manager.cc:5495] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e213c59a73a4213b0e79994f84ec709 (127.10.247.193). New cstate: current_term: 1 leader_uuid: "4e213c59a73a4213b0e79994f84ec709" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e213c59a73a4213b0e79994f84ec709" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 45495 } health_report { overall_health: HEALTHY } } }
I20260928 16:42:09.138302 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:09.132367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.10.247.193:45495)
0928 16:42:09.132585 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:09.132591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:09.132635 (+ 44us) server_negotiation.cc:407] Connection header received
0928 16:42:09.132750 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:09.132754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:09.132856 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:09.132994 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:09.133429 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.134284 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.135744 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.136266 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.136387 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:09.136937 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:09.137696 (+ 759us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:09.137849 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:09.137896 (+ 47us) server_negotiation.cc:299] Negotiation successful
0928 16:42:09.137984 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:42:09.149230 31076 raft_consensus.cc:1261] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [term 1 FOLLOWER]: Refusing update from remote peer 4e213c59a73a4213b0e79994f84ec709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:09.149230 30942 raft_consensus.cc:1261] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [term 1 FOLLOWER]: Refusing update from remote peer 4e213c59a73a4213b0e79994f84ec709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:09.150750 31151 consensus_queue.cc:1035] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "692a89e1f1624d9cb5206cd7e0b9eec6" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:09.151247 31147 consensus_queue.cc:1035] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "401014e9a55746ff96804deed3897cb1" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 34573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:09.171267 31154 mvcc.cc:205] Tried to move back new op lower bound from 7334353834580557824 to 7334353834273697792. Current Snapshot: MvccSnapshot[applied={T|T < 7334353834580557824}]
I20260928 16:42:09.175190 31155 mvcc.cc:205] Tried to move back new op lower bound from 7334353834580557824 to 7334353834273697792. Current Snapshot: MvccSnapshot[applied={T|T < 7334353834580557824}]
I20260928 16:42:09.183562 31156 mvcc.cc:205] Tried to move back new op lower bound from 7334353834580557824 to 7334353834273697792. Current Snapshot: MvccSnapshot[applied={T|T < 7334353834580557824}]
I20260928 16:42:09.193202 30663 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:42:09.193581 30663 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:42:09.198312 30663 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3750963bd2b944dbb03b3216601981bb: Sending DeleteTablet for 3 replicas of tablet 26d629b6f8a340ac8a3e383236a743cf
I20260928 16:42:09.199779 30788 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d629b6f8a340ac8a3e383236a743cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:09 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:43926
I20260928 16:42:09.199774 30922 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d629b6f8a340ac8a3e383236a743cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:09 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:60340
I20260928 16:42:09.199777 31056 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d629b6f8a340ac8a3e383236a743cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:09 UTC) from {username='kudu', principal='kudu/127.10.247.254@KRBTEST.COM'} at 127.0.0.1:41038
I20260928 16:42:09.203226 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 30725
I20260928 16:42:09.216022 31165 tablet_replica.cc:330] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: stopping tablet replica
I20260928 16:42:09.217306 31164 tablet_replica.cc:330] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: stopping tablet replica
I20260928 16:42:09.218055 31164 raft_consensus.cc:2229] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:42:09.218531 31165 raft_consensus.cc:2229] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:42:09.219327 31164 raft_consensus.cc:2258] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:09.219913 31165 raft_consensus.cc:2258] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:09.221897 31166 tablet_replica.cc:330] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: stopping tablet replica
I20260928 16:42:09.222558 31166 raft_consensus.cc:2229] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:42:09.223475 31166 raft_consensus.cc:2258] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:09.224824 31164 ts_tablet_manager.cc:1905] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:09.225143 31165 ts_tablet_manager.cc:1905] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:09.225929 31166 ts_tablet_manager.cc:1905] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 16:42:09 dist-test-slave-9fn6 krb5kdc[30619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.193@KRBTEST.COM
I20260928 16:42:09.234372 31166 ts_tablet_manager.cc:1918] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:09.234647 31166 log.cc:1196] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-0/wal/wals/26d629b6f8a340ac8a3e383236a743cf
I20260928 16:42:09.235337 31166 ts_tablet_manager.cc:1939] T 26d629b6f8a340ac8a3e383236a743cf P 4e213c59a73a4213b0e79994f84ec709: Deleting consensus metadata
I20260928 16:42:09.237993 30650 catalog_manager.cc:4841] TS 4e213c59a73a4213b0e79994f84ec709 (127.10.247.193:45495): tablet 26d629b6f8a340ac8a3e383236a743cf (table test-table [id=148b3d6f099a4456bae31656fcf5b3de]) successfully deleted
I20260928 16:42:09.240072 31165 ts_tablet_manager.cc:1918] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:09.240322 31165 log.cc:1196] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/26d629b6f8a340ac8a3e383236a743cf
I20260928 16:42:09.240646 31164 ts_tablet_manager.cc:1918] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:09.240911 31164 log.cc:1196] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/26d629b6f8a340ac8a3e383236a743cf
I20260928 16:42:09.241003 31165 ts_tablet_manager.cc:1939] T 26d629b6f8a340ac8a3e383236a743cf P 692a89e1f1624d9cb5206cd7e0b9eec6: Deleting consensus metadata
I20260928 16:42:09.241653 31164 ts_tablet_manager.cc:1939] T 26d629b6f8a340ac8a3e383236a743cf P 401014e9a55746ff96804deed3897cb1: Deleting consensus metadata
I20260928 16:42:09.242264 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:09.204570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.10.247.193:45495)
0928 16:42:09.204814 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:09.204823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:09.219573 (+ 14750us) server_negotiation.cc:407] Connection header received
0928 16:42:09.220284 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:09.220291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:09.220566 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:09.220739 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:09.222909 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.224202 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.226158 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.226885 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.226992 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:09.233596 (+ 6604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:09.233619 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:09.233644 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:09.233722 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:09.238193 (+ 4471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:09.239163 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:09.239178 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:09.239182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:09.239390 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:09.239895 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:09.239900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:09.239904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:09.240446 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:09.240842 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:09.241532 (+ 690us) server_negotiation.cc:299] Negotiation successful
0928 16:42:09.241848 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260928 16:42:09.243454 30650 catalog_manager.cc:4841] TS 692a89e1f1624d9cb5206cd7e0b9eec6 (127.10.247.195:42025): tablet 26d629b6f8a340ac8a3e383236a743cf (table test-table [id=148b3d6f099a4456bae31656fcf5b3de]) successfully deleted
I20260928 16:42:09.244324 30848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:09.244365 30651 catalog_manager.cc:4841] TS 401014e9a55746ff96804deed3897cb1 (127.10.247.194:34573): tablet 26d629b6f8a340ac8a3e383236a743cf (table test-table [id=148b3d6f099a4456bae31656fcf5b3de]) successfully deleted
I20260928 16:42:09.354348 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 30725
I20260928 16:42:09.373126 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 30859
Sep 28 16:42:09 dist-test-slave-9fn6 krb5kdc[30619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.194@KRBTEST.COM
I20260928 16:42:09.389578 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:09.374044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.10.247.194:34573)
0928 16:42:09.374263 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:09.374271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:09.374358 (+ 87us) server_negotiation.cc:407] Connection header received
0928 16:42:09.374582 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:09.374587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:09.374663 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:09.374757 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:09.376180 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.377061 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.378402 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.378836 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.378955 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:09.383629 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:09.383653 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:09.383674 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:09.383724 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:09.386766 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:09.387548 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:09.387557 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:09.387560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:09.387707 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:09.388050 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:09.388055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:09.388058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:09.388509 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:09.388837 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:09.389051 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 16:42:09.389248 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260928 16:42:09.390287 30982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:09.486052 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 30859
I20260928 16:42:09.502935 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 30993
Sep 28 16:42:09 dist-test-slave-9fn6 krb5kdc[30619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.195@KRBTEST.COM
I20260928 16:42:09.528282 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:09.503779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41054 (local address 127.10.247.195:42025)
0928 16:42:09.503956 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:09.503962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:09.504018 (+ 56us) server_negotiation.cc:407] Connection header received
0928 16:42:09.504210 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:09.504214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:09.504291 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:09.504393 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:09.505819 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.506617 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.507938 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:09.508343 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:09.508437 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:09.522008 (+ 13571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:09.522033 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:09.522055 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:09.522112 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:09.525358 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:09.526206 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:09.526215 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:09.526218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:09.526383 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:09.526716 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:09.526720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:09.526722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:09.527187 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:09.527447 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:09.527709 (+ 262us) server_negotiation.cc:299] Negotiation successful
0928 16:42:09.527918 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 16:42:09.529105 31116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:09.625006 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 30993
I20260928 16:42:09.642241 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 30632
I20260928 16:42:09.643584 30694 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:09.769616 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 30632
2026-09-28T16:42:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5002 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-28T16:42:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:09Z Disabled control of system clock
I20260928 16:42:09.862571 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:39561
--webserver_interface=127.10.247.254
--webserver_port=0
--builtin_ntp_servers=127.10.247.212:39571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:39561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:42:10.275393 31175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:10.275813 31175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:10.275933 31175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:10.284073 31175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:10.284228 31175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:10.284272 31175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:10.284363 31175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:10.284427 31175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:10.295382 31175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:39561
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:39561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:10.297278 31175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:10.299582 31175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:10.312812 31180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:10.312903 31183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:10.312903 31181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:10.314146 31175 server_base.cc:1034] running on GCE node
I20260928 16:42:10.315084 31175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:10.316883 31175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:10.318099 31175 hybrid_clock.cc:648] HybridClock initialized: now 1790613730318039 us; error 51 us; skew 500 ppm
I20260928 16:42:10.318490 31175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:42:10.321216 31175 webserver.cc:460] Webserver started at http://127.10.247.254:41135/ using document root <none> and password file <none>
I20260928 16:42:10.322072 31175 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:10.322188 31175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:10.322538 31175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:10.326110 31175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "74e812bbca6a4b5ea57c903e50e91e56"
format_stamp: "Formatted at 2026-09-28 16:42:10 on dist-test-slave-9fn6"
server_key: "a6428e1e45144dadd44b992d9723c3e7"
server_key_iv: "56dd7c334a5c494c21cdbcd22ff9944c"
server_key_version: "encryptionkey@0"
I20260928 16:42:10.327013 31175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "74e812bbca6a4b5ea57c903e50e91e56"
format_stamp: "Formatted at 2026-09-28 16:42:10 on dist-test-slave-9fn6"
server_key: "a6428e1e45144dadd44b992d9723c3e7"
server_key_iv: "56dd7c334a5c494c21cdbcd22ff9944c"
server_key_version: "encryptionkey@0"
I20260928 16:42:10.333240 31175 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260928 16:42:10.337294 31189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:10.338894 31175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:42:10.339087 31175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "74e812bbca6a4b5ea57c903e50e91e56"
format_stamp: "Formatted at 2026-09-28 16:42:10 on dist-test-slave-9fn6"
server_key: "a6428e1e45144dadd44b992d9723c3e7"
server_key_iv: "56dd7c334a5c494c21cdbcd22ff9944c"
server_key_version: "encryptionkey@0"
I20260928 16:42:10.339315 31175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:10.380738 31175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:10.381875 31175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:10.382225 31175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:10.402081 31175 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:39561
I20260928 16:42:10.402102 31240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:39561 every 8 connection(s)
I20260928 16:42:10.403687 31175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:10.408458 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 31175
I20260928 16:42:10.408519 31241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:10.408706 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:10.409363 11231 external_mini_cluster.cc:1428] Setting key 8c68a4346f3e6787fe61b307bd09e9cd
I20260928 16:42:10.424288 31241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56: Bootstrap starting.
I20260928 16:42:10.425956 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:10.411714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.10.247.254:39561)
0928 16:42:10.412991 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:10.413004 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:10.413067 (+ 63us) server_negotiation.cc:407] Connection header received
0928 16:42:10.415713 (+ 2646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:10.415744 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:10.416391 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:10.416868 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:10.417520 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:10.418894 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:10.420885 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:10.421681 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:10.421824 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:10.423214 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:10.423223 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:10.423241 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:10.423895 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:10.424007 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:10.424032 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:10.424182 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:10.424407 (+ 225us) server_negotiation.cc:299] Negotiation successful
0928 16:42:10.424809 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":442,"threads_started":1}
I20260928 16:42:10.428373 31241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:10.429755 31241 log.cc:824] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:10.433418 31241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56: No bootstrap required, opened a new log
I20260928 16:42:10.446208 31241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e812bbca6a4b5ea57c903e50e91e56" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39561 } }
I20260928 16:42:10.447010 31241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:10.447187 31241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e812bbca6a4b5ea57c903e50e91e56, State: Initialized, Role: FOLLOWER
I20260928 16:42:10.448298 31241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e812bbca6a4b5ea57c903e50e91e56" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39561 } }
I20260928 16:42:10.448670 31241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:10.448837 31241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:10.449195 31241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:10.453416 31241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e812bbca6a4b5ea57c903e50e91e56" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39561 } }
I20260928 16:42:10.454272 31241 leader_election.cc:302] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74e812bbca6a4b5ea57c903e50e91e56; no voters:
I20260928 16:42:10.455287 31241 leader_election.cc:288] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:10.455439 31246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:10.458384 31241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:10.463174 31246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [term 1 LEADER]: Becoming Leader. State: Replica: 74e812bbca6a4b5ea57c903e50e91e56, State: Running, Role: LEADER
I20260928 16:42:10.463989 31246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e812bbca6a4b5ea57c903e50e91e56" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39561 } }
I20260928 16:42:10.467267 31247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74e812bbca6a4b5ea57c903e50e91e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e812bbca6a4b5ea57c903e50e91e56" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39561 } } }
I20260928 16:42:10.467796 31247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:10.469307 31248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e812bbca6a4b5ea57c903e50e91e56. Latest consensus state: current_term: 1 leader_uuid: "74e812bbca6a4b5ea57c903e50e91e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e812bbca6a4b5ea57c903e50e91e56" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 39561 } } }
I20260928 16:42:10.469828 31248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56 [sys.catalog]: This master's current role is: LEADER
W20260928 16:42:10.476243 31261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:42:10.476403 31261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:42:10.477382 31260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:10.479807 31260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:10.492327 31260 catalog_manager.cc:1347] Generated new cluster ID: 1f46fd758705405e97d5d3b4f2efc35d
I20260928 16:42:10.492463 31260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:10.507272 31260 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:10.508885 31260 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:10.515799 31260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56: Generated new TSK 0
I20260928 16:42:10.517216 31260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:10.536868 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.193:0
--local_ip_for_outbound_sockets=127.10.247.193
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:39561
--builtin_ntp_servers=127.10.247.212:39571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:42:10.939184 31265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:10.939556 31265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:10.939680 31265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:10.947881 31265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:10.948024 31265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:10.948187 31265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.193
I20260928 16:42:10.958575 31265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.247.193
--webserver_port=0
--tserver_master_addrs=127.10.247.254:39561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.193
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:10.960367 31265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:10.962735 31265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:10.978274 31270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:10.978497 31271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:10.978523 31273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:10.979302 31265 server_base.cc:1034] running on GCE node
I20260928 16:42:10.980095 31265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:10.981441 31265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:10.982705 31265 hybrid_clock.cc:648] HybridClock initialized: now 1790613730982638 us; error 51 us; skew 500 ppm
I20260928 16:42:10.983155 31265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:42:10.986125 31265 webserver.cc:460] Webserver started at http://127.10.247.193:36089/ using document root <none> and password file <none>
I20260928 16:42:10.987056 31265 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:10.987177 31265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:10.987560 31265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:10.991429 31265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/data/instance:
uuid: "6e6252a2638d4aaabf495c9d2c7255eb"
format_stamp: "Formatted at 2026-09-28 16:42:10 on dist-test-slave-9fn6"
server_key: "e7135cf0d969b6f6543685d006565896"
server_key_iv: "2e2451c8617eb65ed8d8fa1873ea872b"
server_key_version: "encryptionkey@0"
I20260928 16:42:10.992456 31265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance:
uuid: "6e6252a2638d4aaabf495c9d2c7255eb"
format_stamp: "Formatted at 2026-09-28 16:42:10 on dist-test-slave-9fn6"
server_key: "e7135cf0d969b6f6543685d006565896"
server_key_iv: "2e2451c8617eb65ed8d8fa1873ea872b"
server_key_version: "encryptionkey@0"
I20260928 16:42:10.999428 31265 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260928 16:42:11.004366 31279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:11.006345 31265 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:42:11.006618 31265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/wal
uuid: "6e6252a2638d4aaabf495c9d2c7255eb"
format_stamp: "Formatted at 2026-09-28 16:42:10 on dist-test-slave-9fn6"
server_key: "e7135cf0d969b6f6543685d006565896"
server_key_iv: "2e2451c8617eb65ed8d8fa1873ea872b"
server_key_version: "encryptionkey@0"
I20260928 16:42:11.006899 31265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:11.026718 31265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:11.027863 31265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:11.028220 31265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:11.029489 31265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:11.031374 31265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:11.031486 31265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:11.031618 31265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:11.031715 31265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:11.066751 31265 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.193:39355
I20260928 16:42:11.066792 31391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.193:39355 every 8 connection(s)
I20260928 16:42:11.068373 31265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/data/info.pb
I20260928 16:42:11.078514 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 31265
I20260928 16:42:11.078712 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-0/wal/instance
I20260928 16:42:11.079221 11231 external_mini_cluster.cc:1428] Setting key cd3976daf3439cdc7e1caffa2c7c72bc
I20260928 16:42:11.082901 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.194:0
--local_ip_for_outbound_sockets=127.10.247.194
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:39561
--builtin_ntp_servers=127.10.247.212:39571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:42:11.083071 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:11.071494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:35219 (local address 127.10.247.254:39561)
0928 16:42:11.072036 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:11.072051 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:11.073526 (+ 1475us) server_negotiation.cc:407] Connection header received
0928 16:42:11.075625 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:11.075631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:11.075732 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:11.075845 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:11.077222 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:11.078040 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:11.079906 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:11.080374 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:11.080507 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:11.082198 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:11.082203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:11.082207 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:11.082228 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:11.082277 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:11.082289 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:11.082362 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:11.082625 (+ 263us) server_negotiation.cc:299] Negotiation successful
0928 16:42:11.082724 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":260,"threads_started":1}
I20260928 16:42:11.084870 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:11.072214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:39561 (local address 127.10.247.193:35219)
0928 16:42:11.073323 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:11.073380 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:11.075290 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:11.076130 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:11.076141 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:11.076253 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:11.077033 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:11.077054 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:11.078230 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:11.078234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:11.079743 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:11.079758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:11.080596 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:11.080601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:11.080892 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:11.081775 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:42:11.081779 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:11.081842 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:42:11.081974 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:11.082384 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:11.082396 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:11.082405 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 16:42:11.082617 (+ 212us) client_negotiation.cc:240] Negotiation successful
0928 16:42:11.083148 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":350,"threads_started":1}
I20260928 16:42:11.086822 31392 heartbeater.cc:347] Connected to a master server at 127.10.247.254:39561
I20260928 16:42:11.087219 31392 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:11.088227 31392 heartbeater.cc:511] Master 127.10.247.254:39561 requested a full tablet report, sending...
I20260928 16:42:11.091006 31206 ts_manager.cc:194] Registered new tserver with Master: 6e6252a2638d4aaabf495c9d2c7255eb (127.10.247.193:39355)
I20260928 16:42:11.092993 31206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.193:35219
W20260928 16:42:11.476065 31397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:11.476434 31397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:11.476557 31397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:11.484632 31397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:11.484799 31397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:11.484989 31397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.194
I20260928 16:42:11.495566 31397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.247.194
--webserver_port=0
--tserver_master_addrs=127.10.247.254:39561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.194
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:11.497480 31397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:11.500221 31397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:11.516332 31405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:11.516559 31402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:11.516291 31403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:11.516809 31397 server_base.cc:1034] running on GCE node
I20260928 16:42:11.517791 31397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:11.518930 31397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:11.520145 31397 hybrid_clock.cc:648] HybridClock initialized: now 1790613731520088 us; error 46 us; skew 500 ppm
I20260928 16:42:11.520529 31397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:42:11.523250 31397 webserver.cc:460] Webserver started at http://127.10.247.194:41215/ using document root <none> and password file <none>
I20260928 16:42:11.524097 31397 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:11.524220 31397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:11.524573 31397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:11.528234 31397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/data/instance:
uuid: "546399e9109941d2b31c5eee282ec85a"
format_stamp: "Formatted at 2026-09-28 16:42:11 on dist-test-slave-9fn6"
server_key: "df2ec6f9e441671fc2c01fb518374f52"
server_key_iv: "072eaf1dc647ff2b1fb97831f1d5d300"
server_key_version: "encryptionkey@0"
I20260928 16:42:11.529227 31397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance:
uuid: "546399e9109941d2b31c5eee282ec85a"
format_stamp: "Formatted at 2026-09-28 16:42:11 on dist-test-slave-9fn6"
server_key: "df2ec6f9e441671fc2c01fb518374f52"
server_key_iv: "072eaf1dc647ff2b1fb97831f1d5d300"
server_key_version: "encryptionkey@0"
I20260928 16:42:11.535727 31397 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260928 16:42:11.539878 31411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:11.541560 31397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 16:42:11.541743 31397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/wal
uuid: "546399e9109941d2b31c5eee282ec85a"
format_stamp: "Formatted at 2026-09-28 16:42:11 on dist-test-slave-9fn6"
server_key: "df2ec6f9e441671fc2c01fb518374f52"
server_key_iv: "072eaf1dc647ff2b1fb97831f1d5d300"
server_key_version: "encryptionkey@0"
I20260928 16:42:11.541975 31397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:11.580832 31397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:11.581945 31397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:11.582305 31397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:11.583526 31397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:11.585382 31397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:11.585505 31397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:11.585625 31397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:11.585711 31397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:11.619666 31397 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.194:39137
I20260928 16:42:11.619710 31523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.194:39137 every 8 connection(s)
I20260928 16:42:11.621309 31397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/data/info.pb
I20260928 16:42:11.628445 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 31397
I20260928 16:42:11.628687 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/wal/instance
I20260928 16:42:11.629383 11231 external_mini_cluster.cc:1428] Setting key f504ecd3ce6b4d35e8ea359f321d6578
I20260928 16:42:11.633466 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.247.195:0
--local_ip_for_outbound_sockets=127.10.247.195
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:39561
--builtin_ntp_servers=127.10.247.212:39571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:42:11.637466 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:11.624361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.194:38135 (local address 127.10.247.254:39561)
0928 16:42:11.624992 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:11.624997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:11.626358 (+ 1361us) server_negotiation.cc:407] Connection header received
0928 16:42:11.628505 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:11.628512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:11.628644 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:11.628777 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:11.630510 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:11.631423 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:11.633853 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:11.634387 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:11.634504 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:11.636471 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:11.636476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:11.636480 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:11.636507 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:11.636569 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:11.636584 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:11.636725 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:11.636975 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 16:42:11.637138 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":328,"threads_started":1}
I20260928 16:42:11.639099 31527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:11.625094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:39561 (local address 127.10.247.194:38135)
0928 16:42:11.626170 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:11.626214 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:11.628173 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:11.629139 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:11.629155 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:11.629316 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:11.630291 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:11.630319 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:11.631598 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:11.631604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:11.633608 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:11.633638 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:11.634663 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:11.634671 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:11.635044 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:11.636015 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:42:11.636019 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:11.636087 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:42:11.636231 (+ 144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:11.636725 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:11.636738 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:11.636747 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 16:42:11.636959 (+ 212us) client_negotiation.cc:240] Negotiation successful
0928 16:42:11.637468 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":304,"threads_started":1}
I20260928 16:42:11.640945 31524 heartbeater.cc:347] Connected to a master server at 127.10.247.254:39561
I20260928 16:42:11.641371 31524 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:11.642346 31524 heartbeater.cc:511] Master 127.10.247.254:39561 requested a full tablet report, sending...
I20260928 16:42:11.644150 31206 ts_manager.cc:194] Registered new tserver with Master: 546399e9109941d2b31c5eee282ec85a (127.10.247.194:39137)
I20260928 16:42:11.645387 31206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.194:38135
W20260928 16:42:12.033131 31528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:12.033530 31528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:12.033641 31528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:12.042428 31528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:12.042577 31528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:12.042704 31528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.247.195
I20260928 16:42:12.053395 31528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.247.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.247.195
--webserver_port=0
--tserver_master_addrs=127.10.247.254:39561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.247.195
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:12.055229 31528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:12.057725 31528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:12.073726 31534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:12.073696 31533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:12.074070 31536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:12.074968 31528 server_base.cc:1034] running on GCE node
I20260928 16:42:12.075683 31528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:12.076884 31528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:12.078122 31528 hybrid_clock.cc:648] HybridClock initialized: now 1790613732078064 us; error 45 us; skew 500 ppm
I20260928 16:42:12.078480 31528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:42:12.081103 31528 webserver.cc:460] Webserver started at http://127.10.247.195:41457/ using document root <none> and password file <none>
I20260928 16:42:12.081914 31528 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:12.082020 31528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:12.082367 31528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:12.085795 31528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/data/instance:
uuid: "0c9d7379cf7d4de985e12b926fd6a993"
format_stamp: "Formatted at 2026-09-28 16:42:12 on dist-test-slave-9fn6"
server_key: "efef505c30c9662221c834a743bb8282"
server_key_iv: "0c1af7941d3af21486cc1f332508cf13"
server_key_version: "encryptionkey@0"
I20260928 16:42:12.086706 31528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance:
uuid: "0c9d7379cf7d4de985e12b926fd6a993"
format_stamp: "Formatted at 2026-09-28 16:42:12 on dist-test-slave-9fn6"
server_key: "efef505c30c9662221c834a743bb8282"
server_key_iv: "0c1af7941d3af21486cc1f332508cf13"
server_key_version: "encryptionkey@0"
I20260928 16:42:12.093066 31528 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:42:12.096983 31392 heartbeater.cc:503] Master 127.10.247.254:39561 was elected leader, sending a full tablet report...
I20260928 16:42:12.097743 31542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:12.099783 31528 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:42:12.099969 31528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/wal
uuid: "0c9d7379cf7d4de985e12b926fd6a993"
format_stamp: "Formatted at 2026-09-28 16:42:12 on dist-test-slave-9fn6"
server_key: "efef505c30c9662221c834a743bb8282"
server_key_iv: "0c1af7941d3af21486cc1f332508cf13"
server_key_version: "encryptionkey@0"
I20260928 16:42:12.100222 31528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:12.126868 31528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:12.128049 31528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:12.128435 31528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:12.130060 31528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:42:12.131982 31528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:42:12.132084 31528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:12.132180 31528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:42:12.132239 31528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:12.166577 31528 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.195:43373
I20260928 16:42:12.166635 31654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.195:43373 every 8 connection(s)
I20260928 16:42:12.168219 31528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/data/info.pb
I20260928 16:42:12.169697 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 31528
I20260928 16:42:12.169879 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/wal/instance
I20260928 16:42:12.170470 11231 external_mini_cluster.cc:1428] Setting key c5c57a761ae34c080be21e8d6991a8a8
I20260928 16:42:12.185393 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.171432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.195:42125 (local address 127.10.247.254:39561)
0928 16:42:12.172252 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.172259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.173872 (+ 1613us) server_negotiation.cc:407] Connection header received
0928 16:42:12.176106 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.176111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.176207 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.176333 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:12.178049 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.179270 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.181584 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.182149 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.182281 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.184126 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:12.184132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:12.184137 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:12.184187 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:12.184245 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:12.184261 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:12.184356 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.184744 (+ 388us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.184896 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":531,"threads_started":1}
I20260928 16:42:12.188292 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.172386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:39561 (local address 127.10.247.195:42125)
0928 16:42:12.173699 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.173761 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:12.175695 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:12.176749 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:12.176768 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:12.176917 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:12.177789 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.177816 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.179461 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.179466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.181366 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.181389 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.182452 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.182458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.182740 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.183633 (+ 893us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:42:12.183636 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:12.183697 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:42:12.183820 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:12.184393 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:12.184404 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:12.184412 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 16:42:12.185763 (+ 1351us) client_negotiation.cc:240] Negotiation successful
0928 16:42:12.186405 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":397,"threads_started":1}
I20260928 16:42:12.189874 31655 heartbeater.cc:347] Connected to a master server at 127.10.247.254:39561
I20260928 16:42:12.190246 31655 heartbeater.cc:464] Registering TS with master...
I20260928 16:42:12.191213 31655 heartbeater.cc:511] Master 127.10.247.254:39561 requested a full tablet report, sending...
I20260928 16:42:12.193009 31206 ts_manager.cc:194] Registered new tserver with Master: 0c9d7379cf7d4de985e12b926fd6a993 (127.10.247.195:43373)
I20260928 16:42:12.194034 31206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.247.195:42125
I20260928 16:42:12.200959 11231 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:42:12.213202 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.206038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.10.247.254:39561)
0928 16:42:12.206286 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.206294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.206750 (+ 456us) server_negotiation.cc:407] Connection header received
0928 16:42:12.206952 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.206957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.207046 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.207130 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:12.207695 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.208506 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.210070 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.210613 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.210766 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.212251 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:12.212263 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:12.212268 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:12.212300 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:12.212372 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:12.212395 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:12.212560 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.212685 (+ 125us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.212834 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 16:42:12.218605 31206 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:42:12.226639 31206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:42:12.263258 31671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.241714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35864 (local address 127.10.247.193:39355)
0928 16:42:12.252577 (+ 10863us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.252587 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.252647 (+ 60us) server_negotiation.cc:407] Connection header received
0928 16:42:12.252789 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.252800 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.253219 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.255091 (+ 1872us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:12.255450 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.256871 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.260275 (+ 3404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.261697 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.261944 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.262330 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.262514 (+ 184us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.262734 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10647,"thread_start_us":10493,"threads_started":1}
I20260928 16:42:12.263264 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.243971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:43373 (local address 127.0.0.1:46676)
0928 16:42:12.253939 (+ 9968us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.253966 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:12.254130 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:12.255122 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:12.255127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:12.255151 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:12.255619 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.255631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.257685 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.257691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.260241 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.260254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.262301 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.262309 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.262497 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.262532 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 16:42:12.262603 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 16:42:12.262785 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9813,"mutex_wait_us":84,"thread_start_us":9640,"threads_started":1}
I20260928 16:42:12.263286 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.241335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:39355 (local address 127.0.0.1:35864)
0928 16:42:12.243494 (+ 2159us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.243548 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:12.243741 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:12.254366 (+ 10625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:12.254375 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:12.254422 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:12.254872 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.254886 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.257013 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.257019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.260003 (+ 2984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.260023 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.261950 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.261958 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.262180 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.262244 (+ 64us) client_negotiation.cc:769] Sending connection context
0928 16:42:12.262437 (+ 193us) client_negotiation.cc:240] Negotiation successful
0928 16:42:12.262731 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1865,"mutex_wait_us":64,"thread_start_us":1626,"threads_started":1}
I20260928 16:42:12.263896 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.241335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:39137 (local address 127.0.0.1:34936)
0928 16:42:12.244343 (+ 3008us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.244360 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:12.244484 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:12.255419 (+ 10935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:12.255424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:12.255457 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:12.255896 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.255909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.258024 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.258030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.260528 (+ 2498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.260553 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.263046 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.263052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.263275 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.263307 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 16:42:12.263393 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 16:42:12.263545 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2888,"mutex_wait_us":169,"thread_start_us":3181,"threads_started":1}
I20260928 16:42:12.264883 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.241608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.10.247.194:39137)
0928 16:42:12.254778 (+ 13170us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.254785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.254812 (+ 27us) server_negotiation.cc:407] Connection header received
0928 16:42:12.254910 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.254917 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.255157 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.255976 (+ 819us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:12.256256 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.257832 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.260814 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.262481 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.262663 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.262875 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.264237 (+ 1362us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.264451 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12997,"thread_start_us":12858,"threads_started":1}
I20260928 16:42:12.265163 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.244708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.10.247.195:43373)
0928 16:42:12.254366 (+ 9658us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.254376 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.254419 (+ 43us) server_negotiation.cc:407] Connection header received
0928 16:42:12.254558 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.254568 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.254887 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.255115 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:12.255830 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.257517 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.260464 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.262033 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.263815 (+ 1782us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.264291 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.264374 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.264541 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9462,"thread_start_us":9271,"threads_started":1}
I20260928 16:42:12.268889 31459 tablet_service.cc:1467] Processing CreateTablet for tablet 7c04db3f4059431a96e256f407d5432c (DEFAULT_TABLE table=test-table [id=e33ce414eeaf4cc5b003307a4cf54cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:12.268898 31327 tablet_service.cc:1467] Processing CreateTablet for tablet 7c04db3f4059431a96e256f407d5432c (DEFAULT_TABLE table=test-table [id=e33ce414eeaf4cc5b003307a4cf54cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:12.268896 31590 tablet_service.cc:1467] Processing CreateTablet for tablet 7c04db3f4059431a96e256f407d5432c (DEFAULT_TABLE table=test-table [id=e33ce414eeaf4cc5b003307a4cf54cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:42:12.270366 31459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c04db3f4059431a96e256f407d5432c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:12.270393 31327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c04db3f4059431a96e256f407d5432c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:12.270407 31590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c04db3f4059431a96e256f407d5432c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:12.284817 31674 tablet_bootstrap.cc:492] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: Bootstrap starting.
I20260928 16:42:12.289062 31674 tablet_bootstrap.cc:654] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:12.289462 31675 tablet_bootstrap.cc:492] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: Bootstrap starting.
I20260928 16:42:12.290810 31674 log.cc:824] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:12.292568 31676 tablet_bootstrap.cc:492] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb: Bootstrap starting.
I20260928 16:42:12.293376 31675 tablet_bootstrap.cc:654] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:12.294840 31675 log.cc:824] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:12.296351 31674 tablet_bootstrap.cc:492] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: No bootstrap required, opened a new log
I20260928 16:42:12.296653 31676 tablet_bootstrap.cc:654] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:12.296763 31674 ts_tablet_manager.cc:1397] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260928 16:42:12.298225 31676 log.cc:824] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:12.299532 31675 tablet_bootstrap.cc:492] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: No bootstrap required, opened a new log
I20260928 16:42:12.299881 31675 ts_tablet_manager.cc:1397] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260928 16:42:12.302105 31676 tablet_bootstrap.cc:492] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb: No bootstrap required, opened a new log
I20260928 16:42:12.302471 31676 ts_tablet_manager.cc:1397] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260928 16:42:12.306001 31674 raft_consensus.cc:348] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } }
I20260928 16:42:12.306529 31674 raft_consensus.cc:374] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:12.306639 31674 raft_consensus.cc:729] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546399e9109941d2b31c5eee282ec85a, State: Initialized, Role: FOLLOWER
I20260928 16:42:12.307399 31674 consensus_queue.cc:260] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } }
I20260928 16:42:12.313241 31675 raft_consensus.cc:348] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } }
I20260928 16:42:12.313933 31675 raft_consensus.cc:374] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:12.314054 31675 raft_consensus.cc:729] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9d7379cf7d4de985e12b926fd6a993, State: Initialized, Role: FOLLOWER
I20260928 16:42:12.313774 31676 raft_consensus.cc:348] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } }
I20260928 16:42:12.314277 31676 raft_consensus.cc:374] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:12.314391 31676 raft_consensus.cc:729] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e6252a2638d4aaabf495c9d2c7255eb, State: Initialized, Role: FOLLOWER
I20260928 16:42:12.314949 31675 consensus_queue.cc:260] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } }
I20260928 16:42:12.315069 31676 consensus_queue.cc:260] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } }
I20260928 16:42:12.315281 31524 heartbeater.cc:503] Master 127.10.247.254:39561 was elected leader, sending a full tablet report...
I20260928 16:42:12.316540 31674 ts_tablet_manager.cc:1428] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: Time spent starting tablet: real 0.020s user 0.014s sys 0.003s
I20260928 16:42:12.318188 31655 heartbeater.cc:503] Master 127.10.247.254:39561 was elected leader, sending a full tablet report...
I20260928 16:42:12.318490 31676 ts_tablet_manager.cc:1428] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260928 16:42:12.318749 31675 ts_tablet_manager.cc:1428] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: Time spent starting tablet: real 0.019s user 0.002s sys 0.018s
W20260928 16:42:12.321198 31393 tablet.cc:2369] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:12.334471 31681 raft_consensus.cc:482] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:42:12.334978 31681 raft_consensus.cc:504] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } }
I20260928 16:42:12.337423 31681 leader_election.cc:288] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546399e9109941d2b31c5eee282ec85a (127.10.247.194:39137), 0c9d7379cf7d4de985e12b926fd6a993 (127.10.247.195:43373)
I20260928 16:42:12.346410 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.337910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:54241 (local address 127.10.247.194:39137)
0928 16:42:12.338176 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.338184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.339490 (+ 1306us) server_negotiation.cc:407] Connection header received
0928 16:42:12.339771 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.339776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.339873 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.339984 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:12.340891 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.341664 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.344039 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.345412 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.345600 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.345683 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.345983 (+ 300us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.346117 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:42:12.346503 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.338290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.194:39137 (local address 127.10.247.193:54241)
0928 16:42:12.339344 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.339365 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:12.339600 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:12.339992 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:12.339997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:12.340028 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:12.340714 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.340728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.341805 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.341811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.343801 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.343817 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.345652 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.345658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.345797 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.345834 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 16:42:12.345960 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 16:42:12.346116 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"mutex_wait_us":164,"thread_start_us":585,"threads_started":1}
I20260928 16:42:12.347980 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.338075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.247.193:53331 (local address 127.10.247.195:43373)
0928 16:42:12.338284 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.338289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.339598 (+ 1309us) server_negotiation.cc:407] Connection header received
0928 16:42:12.339783 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.339788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.339870 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.339953 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:42:12.340791 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.341634 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.344146 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.345435 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.345612 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.345683 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.347646 (+ 1963us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.347782 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:42:12.347715 31479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c04db3f4059431a96e256f407d5432c" candidate_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546399e9109941d2b31c5eee282ec85a" is_pre_election: true
I20260928 16:42:12.348093 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.338300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.195:43373 (local address 127.10.247.193:53331)
0928 16:42:12.339455 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.339479 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:12.339639 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:12.340061 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:12.340066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:12.340092 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:42:12.340566 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.340577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.341791 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.341795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.343868 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:12.343884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.347197 (+ 3313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.347205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:12.347393 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.347431 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 16:42:12.347545 (+ 114us) client_negotiation.cc:240] Negotiation successful
0928 16:42:12.347703 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"mutex_wait_us":158,"thread_start_us":504,"threads_started":1}
I20260928 16:42:12.348480 31479 raft_consensus.cc:2454] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e6252a2638d4aaabf495c9d2c7255eb in term 0.
I20260928 16:42:12.349318 31282 leader_election.cc:302] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546399e9109941d2b31c5eee282ec85a, 6e6252a2638d4aaabf495c9d2c7255eb; no voters:
I20260928 16:42:12.349424 31610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c04db3f4059431a96e256f407d5432c" candidate_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c9d7379cf7d4de985e12b926fd6a993" is_pre_election: true
I20260928 16:42:12.349838 31681 raft_consensus.cc:2781] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:42:12.350018 31681 raft_consensus.cc:482] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:42:12.350025 31610 raft_consensus.cc:2454] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e6252a2638d4aaabf495c9d2c7255eb in term 0.
I20260928 16:42:12.350132 31681 raft_consensus.cc:3037] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:12.352912 31681 raft_consensus.cc:504] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } }
I20260928 16:42:12.353612 31681 leader_election.cc:288] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [CANDIDATE]: Term 1 election: Requested vote from peers 546399e9109941d2b31c5eee282ec85a (127.10.247.194:39137), 0c9d7379cf7d4de985e12b926fd6a993 (127.10.247.195:43373)
I20260928 16:42:12.354174 31610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c04db3f4059431a96e256f407d5432c" candidate_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c9d7379cf7d4de985e12b926fd6a993"
I20260928 16:42:12.354184 31479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c04db3f4059431a96e256f407d5432c" candidate_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546399e9109941d2b31c5eee282ec85a"
I20260928 16:42:12.354405 31610 raft_consensus.cc:3037] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:12.354422 31479 raft_consensus.cc:3037] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:12.357046 31610 raft_consensus.cc:2454] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e6252a2638d4aaabf495c9d2c7255eb in term 1.
I20260928 16:42:12.357060 31479 raft_consensus.cc:2454] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e6252a2638d4aaabf495c9d2c7255eb in term 1.
I20260928 16:42:12.357563 31282 leader_election.cc:302] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546399e9109941d2b31c5eee282ec85a, 6e6252a2638d4aaabf495c9d2c7255eb; no voters:
I20260928 16:42:12.357928 31681 raft_consensus.cc:2781] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:12.358585 31681 raft_consensus.cc:686] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [term 1 LEADER]: Becoming Leader. State: Replica: 6e6252a2638d4aaabf495c9d2c7255eb, State: Running, Role: LEADER
I20260928 16:42:12.359254 31681 consensus_queue.cc:237] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } }
I20260928 16:42:12.364948 31205 catalog_manager.cc:5495] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e6252a2638d4aaabf495c9d2c7255eb (127.10.247.193). New cstate: current_term: 1 leader_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e6252a2638d4aaabf495c9d2c7255eb" member_type: VOTER last_known_addr { host: "127.10.247.193" port: 39355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 } health_report { overall_health: UNKNOWN } } }
W20260928 16:42:12.373764 31525 tablet.cc:2369] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:12.416847 31671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.409974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35876 (local address 127.10.247.193:39355)
0928 16:42:12.410282 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.410288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.410374 (+ 86us) server_negotiation.cc:407] Connection header received
0928 16:42:12.410609 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.410613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.410732 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.410879 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:12.411305 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.412052 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.414253 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.414719 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.414823 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.415455 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:12.416169 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:12.416293 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.416426 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.416519 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
W20260928 16:42:12.420154 31656 tablet.cc:2369] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:12.427304 31479 raft_consensus.cc:1261] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [term 1 FOLLOWER]: Refusing update from remote peer 6e6252a2638d4aaabf495c9d2c7255eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:12.427304 31610 raft_consensus.cc:1261] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [term 1 FOLLOWER]: Refusing update from remote peer 6e6252a2638d4aaabf495c9d2c7255eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:12.428383 31685 consensus_queue.cc:1035] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c9d7379cf7d4de985e12b926fd6a993" member_type: VOTER last_known_addr { host: "127.10.247.195" port: 43373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:12.428808 31681 consensus_queue.cc:1035] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "546399e9109941d2b31c5eee282ec85a" member_type: VOTER last_known_addr { host: "127.10.247.194" port: 39137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:12.459089 31205 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:42:12.459474 31205 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:42:12.464277 31205 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74e812bbca6a4b5ea57c903e50e91e56: Sending DeleteTablet for 3 replicas of tablet 7c04db3f4059431a96e256f407d5432c
I20260928 16:42:12.465826 31459 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c04db3f4059431a96e256f407d5432c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:12 UTC) from {username='slave'} at 127.0.0.1:34936
I20260928 16:42:12.465845 31590 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c04db3f4059431a96e256f407d5432c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:12 UTC) from {username='slave'} at 127.0.0.1:46676
I20260928 16:42:12.466733 31327 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c04db3f4059431a96e256f407d5432c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:42:12 UTC) from {username='slave'} at 127.0.0.1:35864
I20260928 16:42:12.467293 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 31265
I20260928 16:42:12.484843 31671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.469150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35890 (local address 127.10.247.193:39355)
0928 16:42:12.469372 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.469380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.475126 (+ 5746us) server_negotiation.cc:407] Connection header received
0928 16:42:12.475361 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.475369 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.475511 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.475654 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:12.476467 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.477836 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.479899 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.480665 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.480776 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.482587 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:12.482602 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:12.482626 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:12.483655 (+ 1029us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:12.483735 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:12.483766 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:12.483864 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.484154 (+ 290us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.484302 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260928 16:42:12.486405 31387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:12.487635 31699 tablet_replica.cc:330] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: stopping tablet replica
I20260928 16:42:12.488343 31699 raft_consensus.cc:2229] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:42:12.489245 31699 raft_consensus.cc:2258] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:12.491441 31698 tablet_replica.cc:330] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: stopping tablet replica
I20260928 16:42:12.492064 31698 raft_consensus.cc:2229] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:42:12.492715 31698 raft_consensus.cc:2258] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:42:12.492995 31699 ts_tablet_manager.cc:1905] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:12.495316 31698 ts_tablet_manager.cc:1905] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:42:12.500957 31698 ts_tablet_manager.cc:1918] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:12.501024 31699 ts_tablet_manager.cc:1918] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:42:12.501153 31698 log.cc:1196] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-2/wal/wals/7c04db3f4059431a96e256f407d5432c
I20260928 16:42:12.501192 31699 log.cc:1196] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: Deleting WAL directory at /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790613388455587-11231-0/minicluster-data/ts-1/wal/wals/7c04db3f4059431a96e256f407d5432c
I20260928 16:42:12.501715 31698 ts_tablet_manager.cc:1939] T 7c04db3f4059431a96e256f407d5432c P 0c9d7379cf7d4de985e12b926fd6a993: Deleting consensus metadata
I20260928 16:42:12.501729 31699 ts_tablet_manager.cc:1939] T 7c04db3f4059431a96e256f407d5432c P 546399e9109941d2b31c5eee282ec85a: Deleting consensus metadata
I20260928 16:42:12.503873 31192 catalog_manager.cc:4841] TS 546399e9109941d2b31c5eee282ec85a (127.10.247.194:39137): tablet 7c04db3f4059431a96e256f407d5432c (table test-table [id=e33ce414eeaf4cc5b003307a4cf54cf3]) successfully deleted
I20260928 16:42:12.503822 31193 catalog_manager.cc:4841] TS 0c9d7379cf7d4de985e12b926fd6a993 (127.10.247.195:43373): tablet 7c04db3f4059431a96e256f407d5432c (table test-table [id=e33ce414eeaf4cc5b003307a4cf54cf3]) successfully deleted
I20260928 16:42:12.600175 31700 tablet_replica.cc:330] T 7c04db3f4059431a96e256f407d5432c P 6e6252a2638d4aaabf495c9d2c7255eb: stopping tablet replica
I20260928 16:42:12.600558 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 31265
W20260928 16:42:12.622359 31193 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.247.193:39355 (error 108)
I20260928 16:42:12.622515 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 31397
I20260928 16:42:12.624725 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.624088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:39355 (local address 127.0.0.1:35906)
0928 16:42:12.624287 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.624444 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:39355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260928 16:42:12.625043 31193 catalog_manager.cc:4584] TS 6e6252a2638d4aaabf495c9d2c7255eb (127.10.247.193:39355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c04db3f4059431a96e256f407d5432c: Network error: Client connection negotiation failed: client connection to 127.10.247.193:39355: connect: Connection refused (error 111)
I20260928 16:42:12.631845 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.623679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34944 (local address 127.10.247.194:39137)
0928 16:42:12.623919 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.623925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.623947 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:42:12.624021 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.624026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.624144 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.624270 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:12.625017 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.626032 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.628160 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.628731 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.628850 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.630187 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:12.630198 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:12.630217 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:12.631001 (+ 784us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:12.631065 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:12.631090 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:12.631195 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.631331 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.631453 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:42:12.632758 31519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:12.679391 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.678642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:39355 (local address 127.0.0.1:35912)
0928 16:42:12.678980 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.679124 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:39355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260928 16:42:12.680751 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.680218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:39355 (local address 127.0.0.1:35924)
0928 16:42:12.680413 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.680517 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:39355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 16:42:12.731699 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 31397
I20260928 16:42:12.741734 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.740981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:39355 (local address 127.0.0.1:35932)
0928 16:42:12.741283 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.741432 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:39355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260928 16:42:12.743022 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.742565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:39355 (local address 127.0.0.1:35942)
0928 16:42:12.742763 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.742864 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:39355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260928 16:42:12.749728 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 31528
I20260928 16:42:12.758355 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.750654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.10.247.195:43373)
0928 16:42:12.750853 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:12.750858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:12.750898 (+ 40us) server_negotiation.cc:407] Connection header received
0928 16:42:12.751187 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:12.751194 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:12.751331 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:12.751440 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:12.751963 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.752937 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.754487 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:12.755015 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:12.755147 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:12.756550 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:12.756562 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:12.756581 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:12.757426 (+ 845us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:12.757494 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:12.757528 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:12.757630 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:12.757830 (+ 200us) server_negotiation.cc:299] Negotiation successful
0928 16:42:12.757968 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:42:12.759220 31650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:12.824067 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.823423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:39355 (local address 127.0.0.1:35950)
0928 16:42:12.823734 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.823860 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:39355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260928 16:42:12.825600 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:12.825111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:39355 (local address 127.0.0.1:35956)
0928 16:42:12.825369 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:12.825456 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:39355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260928 16:42:12.857095 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 31528
I20260928 16:42:12.874006 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 31175
I20260928 16:42:12.875432 31236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:13.003218 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:13.002269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.193:39355 (local address 127.0.0.1:35964)
0928 16:42:13.002781 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:13.002913 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.193:39355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319}
I20260928 16:42:13.003628 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 31175
2026-09-28T16:42:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3233 ms)
[----------] 6 tests from EncryptionPolicyTest (24665 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:13 dist-test-slave-9fn6 krb5kdc[31709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:13 dist-test-slave-9fn6 krb5kdc[31709](info): set up 2 sockets
Sep 28 16:42:13 dist-test-slave-9fn6 krb5kdc[31709](info): commencing operation
krb5kdc: starting...
W20260928 16:42:14.081159 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:14 dist-test-slave-9fn6 krb5kdc[31709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:14.409219 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38569
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:14.799744 31722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:14.800129 31722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:14.800266 31722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:14.808063 31722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:14.808247 31722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:14.808331 31722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:14.808422 31722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:14.808485 31722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:14.819849 31722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38569
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:14.821715 31722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:14.823426 31722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:14.837198 31730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:14.837288 31728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:14.837230 31727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:14.838173 31722 server_base.cc:1034] running on GCE node
I20260928 16:42:14.839203 31722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:14.841209 31722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:14.842433 31722 hybrid_clock.cc:648] HybridClock initialized: now 1790613734842375 us; error 54 us; skew 500 ppm
I20260928 16:42:14.842840 31722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:14 dist-test-slave-9fn6 krb5kdc[31709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:14.850733 31722 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:14.852974 31722 webserver.cc:460] Webserver started at http://127.0.0.1:46443/ using document root <none> and password file <none>
I20260928 16:42:14.853935 31722 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:14.854080 31722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:14.854462 31722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:14.858489 31722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "06a0b49182e0459298a039816944f271"
format_stamp: "Formatted at 2026-09-28 16:42:14 on dist-test-slave-9fn6"
server_key: "c6962f82d6dff84186edce49f8d9c964"
server_key_iv: "2aafbc36422d6d8eeb4971fd8e9f644e"
server_key_version: "encryptionkey@0"
I20260928 16:42:14.859498 31722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "06a0b49182e0459298a039816944f271"
format_stamp: "Formatted at 2026-09-28 16:42:14 on dist-test-slave-9fn6"
server_key: "c6962f82d6dff84186edce49f8d9c964"
server_key_iv: "2aafbc36422d6d8eeb4971fd8e9f644e"
server_key_version: "encryptionkey@0"
I20260928 16:42:14.866003 31722 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260928 16:42:14.870195 31737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:14.871789 31722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:42:14.871990 31722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "06a0b49182e0459298a039816944f271"
format_stamp: "Formatted at 2026-09-28 16:42:14 on dist-test-slave-9fn6"
server_key: "c6962f82d6dff84186edce49f8d9c964"
server_key_iv: "2aafbc36422d6d8eeb4971fd8e9f644e"
server_key_version: "encryptionkey@0"
I20260928 16:42:14.872206 31722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:14.916646 31722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:14.921962 31722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:14.922336 31722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:14.943114 31722 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38569
I20260928 16:42:14.943121 31789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38569 every 8 connection(s)
I20260928 16:42:14.944859 31722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:14.949514 31790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:14.953987 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 31722
I20260928 16:42:14.954241 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:14.955180 11231 external_mini_cluster.cc:1428] Setting key ecbc05a8fcf5d26bacc7e463d2f3e34e
I20260928 16:42:14.964798 31790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271: Bootstrap starting.
I20260928 16:42:14.968856 31790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:14.970042 31790 log.cc:824] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271: Log is configured to *not* fsync() on all Append() calls
Sep 28 16:42:14 dist-test-slave-9fn6 krb5kdc[31709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 16:42:14.973685 31790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271: No bootstrap required, opened a new log
I20260928 16:42:14.980788 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:14.957624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46687 (local address 127.0.0.1:38569)
0928 16:42:14.958733 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:14.958747 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:14.958798 (+ 51us) server_negotiation.cc:407] Connection header received
0928 16:42:14.960790 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:14.960822 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:14.961400 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:14.961863 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:14.963610 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:14.965200 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:14.966596 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:14.967356 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:14.967537 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:14.972326 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:14.972334 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:14.972353 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:14.972394 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:14.975637 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:14.976734 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:14.976746 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:14.976753 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:14.977005 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:14.977510 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:14.977513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:14.977515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:14.978154 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:14.978417 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:14.978988 (+ 571us) server_negotiation.cc:299] Negotiation successful
0928 16:42:14.979434 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":855,"thread_start_us":588,"threads_started":1}
I20260928 16:42:14.983286 31790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a0b49182e0459298a039816944f271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38569 } }
I20260928 16:42:14.983709 31790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:14.983820 31790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06a0b49182e0459298a039816944f271, State: Initialized, Role: FOLLOWER
I20260928 16:42:14.984498 31790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a0b49182e0459298a039816944f271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38569 } }
I20260928 16:42:14.984803 31790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:14.984938 31790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:14.985113 31790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:14.985462 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:14.987635 31790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a0b49182e0459298a039816944f271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38569 } }
I20260928 16:42:14.988221 31790 leader_election.cc:302] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06a0b49182e0459298a039816944f271; no voters:
I20260928 16:42:14.988974 31790 leader_election.cc:288] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:14.989102 31796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:14.990427 31796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [term 1 LEADER]: Becoming Leader. State: Replica: 06a0b49182e0459298a039816944f271, State: Running, Role: LEADER
I20260928 16:42:14.991219 31796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a0b49182e0459298a039816944f271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38569 } }
I20260928 16:42:14.991979 31790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:14.994704 31800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06a0b49182e0459298a039816944f271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a0b49182e0459298a039816944f271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38569 } } }
I20260928 16:42:14.995167 31800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:14.997354 31801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06a0b49182e0459298a039816944f271. Latest consensus state: current_term: 1 leader_uuid: "06a0b49182e0459298a039816944f271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a0b49182e0459298a039816944f271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38569 } } }
I20260928 16:42:14.997786 31801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:15.000829 31805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:15.009117 31805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:15.011739 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:14.991300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48117 (local address 127.0.0.1:38569)
0928 16:42:14.991532 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:14.991541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:14.992141 (+ 600us) server_negotiation.cc:407] Connection header received
0928 16:42:14.992409 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:14.992416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:14.992497 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:14.992582 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:14.994560 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:14.995834 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:14.997410 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:14.998103 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:14.998219 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:15.002315 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:15.002320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:15.002324 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:15.002363 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:15.006650 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:15.007741 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:15.007748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:15.007752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:15.007929 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:15.008458 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:15.008464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:15.008468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:15.009036 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:15.010786 (+ 1750us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:15.011013 (+ 227us) server_negotiation.cc:299] Negotiation successful
0928 16:42:15.011262 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260928 16:42:15.021804 31805 catalog_manager.cc:1347] Generated new cluster ID: c0632eefcbe14f43aaf6c6b08406381a
I20260928 16:42:15.021946 31805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:15.041225 31805 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:15.043169 31805 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:15.057508 31805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06a0b49182e0459298a039816944f271: Generated new TSK 0
I20260928 16:42:15.058683 31805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:15.067809 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 31722
I20260928 16:42:15.069280 31785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:15.172384 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 31722
W20260928 16:42:15.191581 31707 connection.cc:535] client connection to 127.0.0.1:38569 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38569): Connection reset by peer (error 104)
2026-09-28T16:42:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2163 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:15 dist-test-slave-9fn6 krb5kdc[31822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:15 dist-test-slave-9fn6 krb5kdc[31822](info): set up 2 sockets
Sep 28 16:42:15 dist-test-slave-9fn6 krb5kdc[31822](info): commencing operation
krb5kdc: starting...
W20260928 16:42:16.233242 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:16 dist-test-slave-9fn6 krb5kdc[31822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:16.553661 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:16.949587 31835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:16.949967 31835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:16.950098 31835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:16.957947 31835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:16.958077 31835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:16.958139 31835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:16.958230 31835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:16.958294 31835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:16.969353 31835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39261
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:16.971148 31835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:16.972909 31835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:16.986680 31841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:16.986750 31840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:16.986737 31843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:16.987521 31835 server_base.cc:1034] running on GCE node
I20260928 16:42:16.988492 31835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:16.990388 31835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:16.991607 31835 hybrid_clock.cc:648] HybridClock initialized: now 1790613736991558 us; error 50 us; skew 500 ppm
I20260928 16:42:16.992005 31835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:16 dist-test-slave-9fn6 krb5kdc[31822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613736, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:16.995982 31835 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:16.997927 31835 webserver.cc:460] Webserver started at http://127.0.0.1:37085/ using document root <none> and password file <none>
I20260928 16:42:16.998782 31835 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:16.998914 31835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:16.999258 31835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:17.002965 31835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "a4a75416672a41cea40af4aae9a76ed1"
format_stamp: "Formatted at 2026-09-28 16:42:16 on dist-test-slave-9fn6"
server_key: "c0b73ace3cf59ecf7e43942d3147faf3"
server_key_iv: "889d56d0e233da08b4e1deec37d428a5"
server_key_version: "encryptionkey@0"
I20260928 16:42:17.003841 31835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "a4a75416672a41cea40af4aae9a76ed1"
format_stamp: "Formatted at 2026-09-28 16:42:16 on dist-test-slave-9fn6"
server_key: "c0b73ace3cf59ecf7e43942d3147faf3"
server_key_iv: "889d56d0e233da08b4e1deec37d428a5"
server_key_version: "encryptionkey@0"
I20260928 16:42:17.010452 31835 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260928 16:42:17.015089 31850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:17.017017 31835 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260928 16:42:17.017230 31835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "a4a75416672a41cea40af4aae9a76ed1"
format_stamp: "Formatted at 2026-09-28 16:42:16 on dist-test-slave-9fn6"
server_key: "c0b73ace3cf59ecf7e43942d3147faf3"
server_key_iv: "889d56d0e233da08b4e1deec37d428a5"
server_key_version: "encryptionkey@0"
I20260928 16:42:17.017470 31835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:17.036772 31835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:17.041679 31835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:17.041998 31835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:17.062857 31835 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39261
I20260928 16:42:17.062870 31901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39261 every 8 connection(s)
I20260928 16:42:17.064692 31835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:17.068560 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 31835
I20260928 16:42:17.068823 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:17.069473 11231 external_mini_cluster.cc:1428] Setting key ea9d10e416dfb4e55469be071b6dd0d9
I20260928 16:42:17.070012 31902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:17.086561 31905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:17.071720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33701 (local address 127.0.0.1:39261)
0928 16:42:17.072711 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:17.072726 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:17.072778 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:42:17.075612 (+ 2834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:17.075647 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:17.076271 (+ 624us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:17.076864 (+ 593us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:17.077625 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:17.079119 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:17.081037 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:17.081829 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:17.082007 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:17.083607 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:17.083617 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:17.083639 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:17.084394 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:17.084513 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:17.084544 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:17.084774 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:17.085029 (+ 255us) server_negotiation.cc:299] Negotiation successful
0928 16:42:17.085273 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":431,"threads_started":1}
I20260928 16:42:17.086694 31902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1: Bootstrap starting.
I20260928 16:42:17.090950 31902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:17.091204 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:17.092238 31902 log.cc:824] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:17.095839 31902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1: No bootstrap required, opened a new log
I20260928 16:42:17.103701 31905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:17.096347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54709 (local address 127.0.0.1:39261)
0928 16:42:17.096584 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:17.096591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:17.097151 (+ 560us) server_negotiation.cc:407] Connection header received
0928 16:42:17.097275 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:17.097280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:17.097379 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:17.097490 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:17.098020 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:17.099280 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:17.100696 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:17.101441 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:17.101586 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:17.102795 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:17.102803 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:17.102808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:17.102836 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:17.102920 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:17.102942 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:17.103074 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:17.103202 (+ 128us) server_negotiation.cc:299] Negotiation successful
0928 16:42:17.103297 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:42:17.106321 31902 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a75416672a41cea40af4aae9a76ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } }
I20260928 16:42:17.106817 31902 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:17.106902 31902 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4a75416672a41cea40af4aae9a76ed1, State: Initialized, Role: FOLLOWER
I20260928 16:42:17.107596 31902 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a75416672a41cea40af4aae9a76ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } }
I20260928 16:42:17.107860 31902 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:17.107976 31902 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:17.108131 31902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:17.110775 31902 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a75416672a41cea40af4aae9a76ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } }
I20260928 16:42:17.111476 31902 leader_election.cc:302] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4a75416672a41cea40af4aae9a76ed1; no voters:
I20260928 16:42:17.112185 31902 leader_election.cc:288] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:17.112329 31912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:17.113077 31912 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [term 1 LEADER]: Becoming Leader. State: Replica: a4a75416672a41cea40af4aae9a76ed1, State: Running, Role: LEADER
I20260928 16:42:17.113724 31912 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a75416672a41cea40af4aae9a76ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } }
I20260928 16:42:17.114658 31902 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:17.116844 31914 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4a75416672a41cea40af4aae9a76ed1. Latest consensus state: current_term: 1 leader_uuid: "a4a75416672a41cea40af4aae9a76ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a75416672a41cea40af4aae9a76ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } } }
I20260928 16:42:17.117288 31914 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:17.118263 31920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:17.118119 31913 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4a75416672a41cea40af4aae9a76ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a75416672a41cea40af4aae9a76ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } } }
I20260928 16:42:17.118561 31913 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:17.126106 31920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:17.138032 31920 catalog_manager.cc:1347] Generated new cluster ID: 1ffb371aece34855aa1657127417db02
I20260928 16:42:17.138190 31920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:17.158154 31920 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:17.159859 31920 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:17.170426 31920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4a75416672a41cea40af4aae9a76ed1: Generated new TSK 0
I20260928 16:42:17.171564 31920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:17.183384 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 31835
W20260928 16:42:17.184893 31897 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33701
W20260928 16:42:17.185757 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 31835: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:17.185956 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 31835
W20260928 16:42:17.203725 31820 connection.cc:535] client connection to 127.0.0.1:39261 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39261): Connection reset by peer (error 104)
2026-09-28T16:42:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2011 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:17 dist-test-slave-9fn6 krb5kdc[31933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:17 dist-test-slave-9fn6 krb5kdc[31933](info): set up 2 sockets
Sep 28 16:42:17 dist-test-slave-9fn6 krb5kdc[31933](info): commencing operation
krb5kdc: starting...
W20260928 16:42:18.245167 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:18 dist-test-slave-9fn6 krb5kdc[31933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:18.564373 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44371
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:18.953734 31946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:18.954102 31946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:18.954238 31946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:18.962061 31946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:18.962217 31946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:18.962293 31946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:18.962383 31946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:18.962443 31946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:18.973431 31946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44371
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:18.975207 31946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:18.976894 31946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:18.990489 31952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:18.990466 31954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:18.990818 31951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:18.991179 31946 server_base.cc:1034] running on GCE node
I20260928 16:42:18.992168 31946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:18.994045 31946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:18.995258 31946 hybrid_clock.cc:648] HybridClock initialized: now 1790613738995212 us; error 41 us; skew 500 ppm
I20260928 16:42:18.995648 31946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:18 dist-test-slave-9fn6 krb5kdc[31933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613738, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:18.999188 31946 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:19.000957 31946 webserver.cc:460] Webserver started at http://127.0.0.1:42497/ using document root <none> and password file <none>
I20260928 16:42:19.001781 31946 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:19.001905 31946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:19.002249 31946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:19.006003 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "bf803fc1293d4885977f8b822b49d37d"
format_stamp: "Formatted at 2026-09-28 16:42:18 on dist-test-slave-9fn6"
server_key: "69b9276e17c8e0f10d479edb3af311d8"
server_key_iv: "3f8c8973bbb247983b2efd51f6967acc"
server_key_version: "encryptionkey@0"
I20260928 16:42:19.006999 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "bf803fc1293d4885977f8b822b49d37d"
format_stamp: "Formatted at 2026-09-28 16:42:18 on dist-test-slave-9fn6"
server_key: "69b9276e17c8e0f10d479edb3af311d8"
server_key_iv: "3f8c8973bbb247983b2efd51f6967acc"
server_key_version: "encryptionkey@0"
I20260928 16:42:19.013803 31946 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:42:19.018258 31961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:19.019973 31946 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:42:19.020177 31946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "bf803fc1293d4885977f8b822b49d37d"
format_stamp: "Formatted at 2026-09-28 16:42:18 on dist-test-slave-9fn6"
server_key: "69b9276e17c8e0f10d479edb3af311d8"
server_key_iv: "3f8c8973bbb247983b2efd51f6967acc"
server_key_version: "encryptionkey@0"
I20260928 16:42:19.020419 31946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:19.027189 31946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:19.031823 31946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:19.032133 31946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:19.051928 31946 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44371
I20260928 16:42:19.051962 32012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44371 every 8 connection(s)
I20260928 16:42:19.053686 31946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:19.057394 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 31946
I20260928 16:42:19.057615 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:19.058238 11231 external_mini_cluster.cc:1428] Setting key 43930d443de2cadb276db4f110d93bf2
I20260928 16:42:19.058933 32013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:19.068169 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:19.060594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46907 (local address 127.0.0.1:44371)
0928 16:42:19.061566 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:19.061580 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:19.061630 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:42:19.064272 (+ 2642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:19.064298 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:19.064768 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:19.065222 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:19.066355 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:19.066366 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:19.066379 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:19.066404 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:19.066469 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:19.066490 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:19.066727 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:19.066867 (+ 140us) server_negotiation.cc:299] Negotiation successful
0928 16:42:19.067071 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":424,"threads_started":1}
I20260928 16:42:19.072507 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:19.074076 32013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d: Bootstrap starting.
I20260928 16:42:19.078161 32013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:19.079797 32013 log.cc:824] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:19.080243 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:19.077489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50319 (local address 127.0.0.1:44371)
0928 16:42:19.077711 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:19.077718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:19.078075 (+ 357us) server_negotiation.cc:407] Connection header received
0928 16:42:19.078297 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:19.078303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:19.078368 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:19.078458 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:19.079537 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:19.079542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:19.079545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:19.079561 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:19.079610 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:19.079626 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:19.079720 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:19.079859 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 16:42:19.079914 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260928 16:42:19.083635 32013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d: No bootstrap required, opened a new log
I20260928 16:42:19.092265 32013 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf803fc1293d4885977f8b822b49d37d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44371 } }
I20260928 16:42:19.092864 32013 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:19.093044 32013 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf803fc1293d4885977f8b822b49d37d, State: Initialized, Role: FOLLOWER
I20260928 16:42:19.093798 32013 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf803fc1293d4885977f8b822b49d37d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44371 } }
I20260928 16:42:19.094074 32013 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:19.094235 32013 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:19.094435 32013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:19.097525 32013 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf803fc1293d4885977f8b822b49d37d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44371 } }
I20260928 16:42:19.098217 32013 leader_election.cc:302] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf803fc1293d4885977f8b822b49d37d; no voters:
I20260928 16:42:19.098914 32013 leader_election.cc:288] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:19.099044 32023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:19.099735 32023 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [term 1 LEADER]: Becoming Leader. State: Replica: bf803fc1293d4885977f8b822b49d37d, State: Running, Role: LEADER
I20260928 16:42:19.100370 32023 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf803fc1293d4885977f8b822b49d37d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44371 } }
I20260928 16:42:19.101563 32013 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:19.103178 32025 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf803fc1293d4885977f8b822b49d37d. Latest consensus state: current_term: 1 leader_uuid: "bf803fc1293d4885977f8b822b49d37d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf803fc1293d4885977f8b822b49d37d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44371 } } }
I20260928 16:42:19.103641 32025 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:19.106971 32024 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf803fc1293d4885977f8b822b49d37d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf803fc1293d4885977f8b822b49d37d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44371 } } }
I20260928 16:42:19.107327 32024 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:19.108863 32030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:19.114799 32030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:19.126160 32030 catalog_manager.cc:1347] Generated new cluster ID: 59a1ec07cc804ffbaedcfa10badaf39e
I20260928 16:42:19.126356 32030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:19.148082 32030 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:19.148671 32030 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:19.157375 32030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf803fc1293d4885977f8b822b49d37d: Generated new TSK 0
I20260928 16:42:19.158474 32030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:19.164180 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 31946
W20260928 16:42:19.165527 32008 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46907
W20260928 16:42:19.166316 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 31946: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:19.166500 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 31946
2026-09-28T16:42:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1980 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:19 dist-test-slave-9fn6 krb5kdc[32045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:19 dist-test-slave-9fn6 krb5kdc[32045](info): set up 2 sockets
Sep 28 16:42:19 dist-test-slave-9fn6 krb5kdc[32045](info): commencing operation
krb5kdc: starting...
W20260928 16:42:20.245242 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.006s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:20 dist-test-slave-9fn6 krb5kdc[32045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:20.584975 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41871
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:20.988549 32058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:20.988991 32058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:20.989111 32058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:20.997485 32058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:20.997684 32058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:20.997742 32058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:20.997809 32058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:20.997856 32058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:21.009709 32058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:21.011518 32058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:21.013242 32058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:21.027877 32063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:21.027873 32066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:21.027899 32064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:21.028877 32058 server_base.cc:1034] running on GCE node
I20260928 16:42:21.030052 32058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:21.031924 32058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:21.033181 32058 hybrid_clock.cc:648] HybridClock initialized: now 1790613741033116 us; error 37 us; skew 500 ppm
I20260928 16:42:21.033608 32058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:21 dist-test-slave-9fn6 krb5kdc[32045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613741, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:21.037441 32058 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:21.039362 32058 webserver.cc:460] Webserver started at http://127.0.0.1:32835/ using document root <none> and password file <none>
I20260928 16:42:21.040220 32058 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:21.040344 32058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:21.040738 32058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:21.044543 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045"
format_stamp: "Formatted at 2026-09-28 16:42:21 on dist-test-slave-9fn6"
server_key: "c6b02a9af61412c409a6a2355ea9a54b"
server_key_iv: "e6fa068ad352fd971b84b706ab24e216"
server_key_version: "encryptionkey@0"
I20260928 16:42:21.045642 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045"
format_stamp: "Formatted at 2026-09-28 16:42:21 on dist-test-slave-9fn6"
server_key: "c6b02a9af61412c409a6a2355ea9a54b"
server_key_iv: "e6fa068ad352fd971b84b706ab24e216"
server_key_version: "encryptionkey@0"
I20260928 16:42:21.052476 32058 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260928 16:42:21.056797 32073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:21.058530 32058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:42:21.058750 32058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045"
format_stamp: "Formatted at 2026-09-28 16:42:21 on dist-test-slave-9fn6"
server_key: "c6b02a9af61412c409a6a2355ea9a54b"
server_key_iv: "e6fa068ad352fd971b84b706ab24e216"
server_key_version: "encryptionkey@0"
I20260928 16:42:21.058983 32058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:21.088538 32058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:21.094233 32058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:21.094569 32058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:21.115293 32058 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41871
I20260928 16:42:21.115317 32124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41871 every 8 connection(s)
I20260928 16:42:21.117000 32058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:21.120559 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 32058
I20260928 16:42:21.120803 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:21.121328 11231 external_mini_cluster.cc:1428] Setting key ec9a00b0dc3e38ee238c881f74838f61
I20260928 16:42:21.121764 32125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:21.138159 32125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045: Bootstrap starting.
Sep 28 16:42:21 dist-test-slave-9fn6 krb5kdc[32045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 16:42:21.142570 32125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:21.144037 32125 log.cc:824] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:21.147435 32125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045: No bootstrap required, opened a new log
I20260928 16:42:21.147785 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:21.123320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37657 (local address 127.0.0.1:41871)
0928 16:42:21.124222 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:21.124236 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:21.124284 (+ 48us) server_negotiation.cc:407] Connection header received
0928 16:42:21.127036 (+ 2752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:21.127066 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:21.127629 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:21.128062 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:21.130298 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:21.131775 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:21.133611 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:21.134459 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:21.134618 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:21.139670 (+ 5052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:21.139679 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:21.139698 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:21.139759 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:21.143047 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:21.144003 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:21.144010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:21.144018 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:21.144233 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:21.144660 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:21.144664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:21.144667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:21.145304 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:21.145652 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:21.146090 (+ 438us) server_negotiation.cc:299] Negotiation successful
0928 16:42:21.146589 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":410,"threads_started":1}
I20260928 16:42:21.152248 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:21.156919 32125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41871 } }
I20260928 16:42:21.157545 32125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:21.157725 32125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fa40cdfcf3e4d5b98de7f0f4867c045, State: Initialized, Role: FOLLOWER
I20260928 16:42:21.158578 32125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41871 } }
I20260928 16:42:21.158932 32125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:21.159097 32125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:21.159303 32125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:21.162671 32125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41871 } }
I20260928 16:42:21.163441 32125 leader_election.cc:302] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fa40cdfcf3e4d5b98de7f0f4867c045; no voters:
I20260928 16:42:21.164214 32125 leader_election.cc:288] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:21.164436 32135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:21.165633 32135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [term 1 LEADER]: Becoming Leader. State: Replica: 9fa40cdfcf3e4d5b98de7f0f4867c045, State: Running, Role: LEADER
I20260928 16:42:21.166288 32135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41871 } }
I20260928 16:42:21.167186 32125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:21.170552 32136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41871 } } }
I20260928 16:42:21.171119 32136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:21.170603 32137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fa40cdfcf3e4d5b98de7f0f4867c045. Latest consensus state: current_term: 1 leader_uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa40cdfcf3e4d5b98de7f0f4867c045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41871 } } }
I20260928 16:42:21.177189 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:21.157328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57323 (local address 127.0.0.1:41871)
0928 16:42:21.157579 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:21.157587 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:21.158171 (+ 584us) server_negotiation.cc:407] Connection header received
0928 16:42:21.158409 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:21.158413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:21.158490 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:21.158570 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:21.160759 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:21.161594 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:21.163349 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:21.163911 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:21.164062 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:21.167544 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:21.167550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:21.167556 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:21.167601 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:21.172152 (+ 4551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:21.173328 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:21.173336 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:21.173340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:21.173525 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:21.174040 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:21.174046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:21.174050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:21.174537 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:21.174871 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:21.176320 (+ 1449us) server_negotiation.cc:299] Negotiation successful
0928 16:42:21.176647 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 16:42:21.176781 32137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:21.179705 32142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:21.186187 32142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:21.196794 32142 catalog_manager.cc:1347] Generated new cluster ID: e1452e540c0f47599a652fdce27b69f0
I20260928 16:42:21.197023 32142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:21.232594 32142 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:21.234301 32142 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:21.243361 32142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fa40cdfcf3e4d5b98de7f0f4867c045: Generated new TSK 0
I20260928 16:42:21.244437 32142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:21.260174 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32058
I20260928 16:42:21.261577 32120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:21.365944 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 32058
2026-09-28T16:42:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2199 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:21 dist-test-slave-9fn6 krb5kdc[32157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:21 dist-test-slave-9fn6 krb5kdc[32157](info): set up 2 sockets
Sep 28 16:42:21 dist-test-slave-9fn6 krb5kdc[32157](info): commencing operation
krb5kdc: starting...
W20260928 16:42:22.429364 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.006s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:22 dist-test-slave-9fn6 krb5kdc[32157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:22.758716 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41711
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:23.160100 32170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:23.160475 32170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:23.160643 32170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:23.168586 32170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:23.168771 32170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:23.168855 32170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:23.168944 32170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:23.169001 32170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:23.180397 32170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:23.182225 32170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:23.183892 32170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:23.197978 32178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:23.198045 32176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:23.198001 32175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:23.199338 32170 server_base.cc:1034] running on GCE node
I20260928 16:42:23.200399 32170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:23.202359 32170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:23.203599 32170 hybrid_clock.cc:648] HybridClock initialized: now 1790613743203548 us; error 39 us; skew 500 ppm
I20260928 16:42:23.204028 32170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:23 dist-test-slave-9fn6 krb5kdc[32157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613743, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:23.207612 32170 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:23.209441 32170 webserver.cc:460] Webserver started at http://127.0.0.1:46595/ using document root <none> and password file <none>
I20260928 16:42:23.210266 32170 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:23.210386 32170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:23.210724 32170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:23.214424 32170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "60a8a29f086a4edb89a7138840ba41ec"
format_stamp: "Formatted at 2026-09-28 16:42:23 on dist-test-slave-9fn6"
server_key: "8e81bd9c0854ca9af88a153725e854ba"
server_key_iv: "f823b8bfb5a88be7dd9a659d364018f9"
server_key_version: "encryptionkey@0"
I20260928 16:42:23.215301 32170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "60a8a29f086a4edb89a7138840ba41ec"
format_stamp: "Formatted at 2026-09-28 16:42:23 on dist-test-slave-9fn6"
server_key: "8e81bd9c0854ca9af88a153725e854ba"
server_key_iv: "f823b8bfb5a88be7dd9a659d364018f9"
server_key_version: "encryptionkey@0"
I20260928 16:42:23.221522 32170 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260928 16:42:23.225598 32186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:23.227192 32170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:42:23.227376 32170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "60a8a29f086a4edb89a7138840ba41ec"
format_stamp: "Formatted at 2026-09-28 16:42:23 on dist-test-slave-9fn6"
server_key: "8e81bd9c0854ca9af88a153725e854ba"
server_key_iv: "f823b8bfb5a88be7dd9a659d364018f9"
server_key_version: "encryptionkey@0"
I20260928 16:42:23.227596 32170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:23.256191 32170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:23.261487 32170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:23.261816 32170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:23.282191 32170 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41711
I20260928 16:42:23.282183 32238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41711 every 8 connection(s)
I20260928 16:42:23.283906 32170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:23.288426 32239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:23.292958 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 32170
I20260928 16:42:23.293146 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:23.293668 11231 external_mini_cluster.cc:1428] Setting key a4ab97b6227ee0b0d2a03f1d0fc27e90
I20260928 16:42:23.303288 32239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec: Bootstrap starting.
I20260928 16:42:23.307370 32239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:23.309021 32239 log.cc:824] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:23.309823 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:23.296028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48987 (local address 127.0.0.1:41711)
0928 16:42:23.297037 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:23.297052 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:23.297101 (+ 49us) server_negotiation.cc:407] Connection header received
0928 16:42:23.299102 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:23.299143 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:23.299751 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:23.300278 (+ 527us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:23.301036 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:23.302563 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:23.304117 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:23.304931 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:23.305149 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:23.306490 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:23.306500 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:23.306518 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:23.307178 (+ 660us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:23.307286 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:23.307318 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:23.307434 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:23.307720 (+ 286us) server_negotiation.cc:299] Negotiation successful
0928 16:42:23.308108 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":448,"threads_started":1}
I20260928 16:42:23.312706 32239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec: No bootstrap required, opened a new log
I20260928 16:42:23.314347 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:23.323835 32239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a8a29f086a4edb89a7138840ba41ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41711 } }
I20260928 16:42:23.324481 32239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:23.324599 32239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a8a29f086a4edb89a7138840ba41ec, State: Initialized, Role: FOLLOWER
I20260928 16:42:23.325433 32239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a8a29f086a4edb89a7138840ba41ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41711 } }
I20260928 16:42:23.325722 32239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:23.325884 32239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:23.326097 32239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:23.327862 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:23.320436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34635 (local address 127.0.0.1:41711)
0928 16:42:23.320746 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:23.320755 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:23.321177 (+ 422us) server_negotiation.cc:407] Connection header received
0928 16:42:23.321479 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:23.321484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:23.321572 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:23.321682 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:23.322402 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:23.323217 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:23.325092 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:23.325645 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:23.325784 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:23.327064 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:23.327069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:23.327072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:23.327099 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:23.327148 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:23.327165 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:23.327277 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:23.327416 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 16:42:23.327517 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260928 16:42:23.329360 32239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a8a29f086a4edb89a7138840ba41ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41711 } }
I20260928 16:42:23.329994 32239 leader_election.cc:302] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60a8a29f086a4edb89a7138840ba41ec; no voters:
I20260928 16:42:23.330621 32239 leader_election.cc:288] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:23.330791 32249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:23.331511 32249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [term 1 LEADER]: Becoming Leader. State: Replica: 60a8a29f086a4edb89a7138840ba41ec, State: Running, Role: LEADER
I20260928 16:42:23.332178 32249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a8a29f086a4edb89a7138840ba41ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41711 } }
I20260928 16:42:23.333269 32239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:23.335475 32251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60a8a29f086a4edb89a7138840ba41ec. Latest consensus state: current_term: 1 leader_uuid: "60a8a29f086a4edb89a7138840ba41ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a8a29f086a4edb89a7138840ba41ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41711 } } }
I20260928 16:42:23.335729 32250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60a8a29f086a4edb89a7138840ba41ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a8a29f086a4edb89a7138840ba41ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41711 } } }
I20260928 16:42:23.335978 32251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:23.336059 32250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:23.338297 32256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:23.344357 32256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:23.355789 32256 catalog_manager.cc:1347] Generated new cluster ID: 0bd0736c891e4af7bbd870afd7fad33f
I20260928 16:42:23.355938 32256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:23.392060 32256 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:23.393741 32256 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:23.409549 32256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60a8a29f086a4edb89a7138840ba41ec: Generated new TSK 0
I20260928 16:42:23.411046 32256 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:23.428915 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32170
W20260928 16:42:23.430347 32234 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48987
W20260928 16:42:23.431178 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32170: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:23.431329 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 32170
2026-09-28T16:42:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2064 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:23 dist-test-slave-9fn6 krb5kdc[32270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:23 dist-test-slave-9fn6 krb5kdc[32270](info): set up 2 sockets
Sep 28 16:42:23 dist-test-slave-9fn6 krb5kdc[32270](info): commencing operation
krb5kdc: starting...
W20260928 16:42:24.481058 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.006s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:24 dist-test-slave-9fn6 krb5kdc[32270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:24.804961 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:25.202637 32283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:25.203118 32283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:25.203289 32283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:25.211474 32283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:25.211629 32283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:25.211694 32283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:25.211784 32283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:25.211841 32283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:25.223235 32283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:25.225114 32283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:25.226751 32283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:25.240756 32288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:25.240729 32291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:25.240778 32289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:25.241797 32283 server_base.cc:1034] running on GCE node
I20260928 16:42:25.242785 32283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:25.245436 32283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:25.246796 32283 hybrid_clock.cc:648] HybridClock initialized: now 1790613745246625 us; error 132 us; skew 500 ppm
I20260928 16:42:25.247340 32283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:25 dist-test-slave-9fn6 krb5kdc[32270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613745, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:25.251215 32283 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:25.253294 32283 webserver.cc:460] Webserver started at http://127.0.0.1:38589/ using document root <none> and password file <none>
I20260928 16:42:25.254194 32283 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:25.254336 32283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:25.254701 32283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:25.258556 32283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "bad045c7e1614e3eba3106159f9e5198"
format_stamp: "Formatted at 2026-09-28 16:42:25 on dist-test-slave-9fn6"
server_key: "412d81ae584fa7e131692fbfd9f2756c"
server_key_iv: "675b985dac85b77c86d061de87f3274c"
server_key_version: "encryptionkey@0"
I20260928 16:42:25.259480 32283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "bad045c7e1614e3eba3106159f9e5198"
format_stamp: "Formatted at 2026-09-28 16:42:25 on dist-test-slave-9fn6"
server_key: "412d81ae584fa7e131692fbfd9f2756c"
server_key_iv: "675b985dac85b77c86d061de87f3274c"
server_key_version: "encryptionkey@0"
I20260928 16:42:25.266448 32283 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260928 16:42:25.271049 32298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:25.272866 32283 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:42:25.273110 32283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "bad045c7e1614e3eba3106159f9e5198"
format_stamp: "Formatted at 2026-09-28 16:42:25 on dist-test-slave-9fn6"
server_key: "412d81ae584fa7e131692fbfd9f2756c"
server_key_iv: "675b985dac85b77c86d061de87f3274c"
server_key_version: "encryptionkey@0"
I20260928 16:42:25.273355 32283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:25.280772 32283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:25.286046 32283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:25.286401 32283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:25.307279 32283 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38911
I20260928 16:42:25.307314 32351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38911 every 8 connection(s)
I20260928 16:42:25.309100 32283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:25.309836 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 32283
I20260928 16:42:25.310045 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:25.310709 11231 external_mini_cluster.cc:1428] Setting key 6b07ab8472658dcb1b430595f3d85f46
I20260928 16:42:25.314451 32352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:25.319834 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:25.312941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55871 (local address 127.0.0.1:38911)
0928 16:42:25.313838 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:25.313851 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:25.313901 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:42:25.315836 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:25.315862 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:25.316272 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:25.316725 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:25.317933 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:25.317940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:25.317954 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:25.317973 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:25.318075 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:25.318102 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:25.318340 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:25.318486 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 16:42:25.318704 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":364,"threads_started":1}
I20260928 16:42:25.324548 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:25.328114 32352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198: Bootstrap starting.
I20260928 16:42:25.332141 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:25.329438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34621 (local address 127.0.0.1:38911)
0928 16:42:25.329651 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:25.329659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:25.330096 (+ 437us) server_negotiation.cc:407] Connection header received
0928 16:42:25.330266 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:25.330271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:25.330346 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:25.330468 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:25.331462 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:25.331469 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:25.331473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:25.331493 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:25.331549 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:25.331572 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:25.331688 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:25.331762 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 16:42:25.331825 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:42:25.332463 32352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:25.333844 32352 log.cc:824] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:25.337217 32352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198: No bootstrap required, opened a new log
I20260928 16:42:25.345492 32352 raft_consensus.cc:348] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad045c7e1614e3eba3106159f9e5198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } }
I20260928 16:42:25.345958 32352 raft_consensus.cc:374] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:25.346065 32352 raft_consensus.cc:729] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad045c7e1614e3eba3106159f9e5198, State: Initialized, Role: FOLLOWER
I20260928 16:42:25.346794 32352 consensus_queue.cc:260] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad045c7e1614e3eba3106159f9e5198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } }
I20260928 16:42:25.347056 32352 raft_consensus.cc:388] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:25.347210 32352 raft_consensus.cc:482] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:25.347420 32352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:25.350045 32352 raft_consensus.cc:504] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad045c7e1614e3eba3106159f9e5198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } }
I20260928 16:42:25.350656 32352 leader_election.cc:302] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bad045c7e1614e3eba3106159f9e5198; no voters:
I20260928 16:42:25.351315 32352 leader_election.cc:288] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:25.351459 32362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:25.352097 32362 raft_consensus.cc:686] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [term 1 LEADER]: Becoming Leader. State: Replica: bad045c7e1614e3eba3106159f9e5198, State: Running, Role: LEADER
I20260928 16:42:25.353871 32352 sys_catalog.cc:564] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:25.353943 32362 consensus_queue.cc:237] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad045c7e1614e3eba3106159f9e5198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } }
I20260928 16:42:25.357297 32364 sys_catalog.cc:455] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bad045c7e1614e3eba3106159f9e5198. Latest consensus state: current_term: 1 leader_uuid: "bad045c7e1614e3eba3106159f9e5198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad045c7e1614e3eba3106159f9e5198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } } }
I20260928 16:42:25.357960 32364 sys_catalog.cc:458] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:25.357852 32363 sys_catalog.cc:455] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bad045c7e1614e3eba3106159f9e5198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad045c7e1614e3eba3106159f9e5198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } } }
I20260928 16:42:25.358129 32363 sys_catalog.cc:458] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198 [sys.catalog]: This master's current role is: LEADER
W20260928 16:42:25.365870 32378 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:42:25.366037 32378 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:42:25.366179 32371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:25.367974 32371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:25.378420 32371 catalog_manager.cc:1347] Generated new cluster ID: 4bb3a447924e45ec8595fa70475d7087
I20260928 16:42:25.378573 32371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:25.415364 32371 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:25.415962 32371 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:25.425048 32371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bad045c7e1614e3eba3106159f9e5198: Generated new TSK 0
I20260928 16:42:25.426421 32371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:25.435877 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32283
W20260928 16:42:25.437312 32347 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55871
W20260928 16:42:25.438169 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32283: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:25.438331 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 32283
2026-09-28T16:42:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2007 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:25 dist-test-slave-9fn6 krb5kdc[32383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:25 dist-test-slave-9fn6 krb5kdc[32383](info): set up 2 sockets
Sep 28 16:42:25 dist-test-slave-9fn6 krb5kdc[32383](info): commencing operation
krb5kdc: starting...
W20260928 16:42:26.501138 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.010s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:26 dist-test-slave-9fn6 krb5kdc[32383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:26.841782 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:27.240784 32396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:27.241202 32396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:27.241367 32396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:27.249624 32396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:27.249769 32396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:27.249825 32396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:27.249914 32396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:27.249975 32396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:27.262534 32396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:27.264319 32396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:27.266078 32396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:27.280331 32401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:27.280424 32402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:27.280424 32404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:27.281584 32396 server_base.cc:1034] running on GCE node
I20260928 16:42:27.282634 32396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:27.284919 32396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:27.286177 32396 hybrid_clock.cc:648] HybridClock initialized: now 1790613747286122 us; error 46 us; skew 500 ppm
I20260928 16:42:27.286640 32396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:27 dist-test-slave-9fn6 krb5kdc[32383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613747, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:27.290618 32396 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:27.292526 32396 webserver.cc:460] Webserver started at http://127.0.0.1:43683/ using document root <none> and password file <none>
I20260928 16:42:27.293417 32396 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:27.293550 32396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:27.293907 32396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:27.297752 32396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "1d0f7050f7af4c3fbc1f606d854c44b8"
format_stamp: "Formatted at 2026-09-28 16:42:27 on dist-test-slave-9fn6"
server_key: "8fdb104969f2c448b063564a09436f3d"
server_key_iv: "060960197b3a0040f10d38c55fee1352"
server_key_version: "encryptionkey@0"
I20260928 16:42:27.298687 32396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "1d0f7050f7af4c3fbc1f606d854c44b8"
format_stamp: "Formatted at 2026-09-28 16:42:27 on dist-test-slave-9fn6"
server_key: "8fdb104969f2c448b063564a09436f3d"
server_key_iv: "060960197b3a0040f10d38c55fee1352"
server_key_version: "encryptionkey@0"
I20260928 16:42:27.305550 32396 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 16:42:27.310348 32411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:27.312141 32396 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:42:27.312395 32396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "1d0f7050f7af4c3fbc1f606d854c44b8"
format_stamp: "Formatted at 2026-09-28 16:42:27 on dist-test-slave-9fn6"
server_key: "8fdb104969f2c448b063564a09436f3d"
server_key_iv: "060960197b3a0040f10d38c55fee1352"
server_key_version: "encryptionkey@0"
I20260928 16:42:27.312667 32396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:27.364825 32396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:27.370293 32396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:27.370646 32396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:27.391888 32396 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46427
I20260928 16:42:27.391904 32463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46427 every 8 connection(s)
I20260928 16:42:27.393594 32396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:27.397773 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 32396
I20260928 16:42:27.397994 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:27.398631 32464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:27.398658 11231 external_mini_cluster.cc:1428] Setting key a5f13a6343d8ee629a497c6023694517
Sep 28 16:42:27 dist-test-slave-9fn6 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 16:42:27.415694 32464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8: Bootstrap starting.
I20260928 16:42:27.420469 32464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:27.422089 32464 log.cc:824] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:27.425904 32464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8: No bootstrap required, opened a new log
I20260928 16:42:27.426425 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:27.400807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:58267 (local address 127.0.0.1:46427)
0928 16:42:27.401695 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:27.401713 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:27.401770 (+ 57us) server_negotiation.cc:407] Connection header received
0928 16:42:27.404088 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:27.404117 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:27.404776 (+ 659us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:27.405213 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:27.407221 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:27.408746 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:27.410645 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:27.411467 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:27.411614 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:27.417293 (+ 5679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:27.417304 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:27.417323 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:27.417375 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:27.420892 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:27.422234 (+ 1342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:27.422246 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:27.422256 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:27.422479 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:27.423088 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:27.423092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:27.423094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:27.423841 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:27.424158 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:27.424753 (+ 595us) server_negotiation.cc:299] Negotiation successful
0928 16:42:27.425201 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":419,"threads_started":1}
I20260928 16:42:27.431252 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:27.435562 32464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0f7050f7af4c3fbc1f606d854c44b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46427 } }
I20260928 16:42:27.436089 32464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:27.436237 32464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d0f7050f7af4c3fbc1f606d854c44b8, State: Initialized, Role: FOLLOWER
I20260928 16:42:27.437101 32464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0f7050f7af4c3fbc1f606d854c44b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46427 } }
I20260928 16:42:27.437448 32464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:27.437629 32464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:27.437836 32464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:27.440919 32464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0f7050f7af4c3fbc1f606d854c44b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46427 } }
I20260928 16:42:27.441532 32464 leader_election.cc:302] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d0f7050f7af4c3fbc1f606d854c44b8; no voters:
I20260928 16:42:27.442291 32464 leader_election.cc:288] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:27.442391 32474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:27.444154 32474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [term 1 LEADER]: Becoming Leader. State: Replica: 1d0f7050f7af4c3fbc1f606d854c44b8, State: Running, Role: LEADER
I20260928 16:42:27.444885 32474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0f7050f7af4c3fbc1f606d854c44b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46427 } }
I20260928 16:42:27.445269 32464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:27.448153 32475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d0f7050f7af4c3fbc1f606d854c44b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0f7050f7af4c3fbc1f606d854c44b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46427 } } }
I20260928 16:42:27.448944 32475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:27.448899 32476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d0f7050f7af4c3fbc1f606d854c44b8. Latest consensus state: current_term: 1 leader_uuid: "1d0f7050f7af4c3fbc1f606d854c44b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0f7050f7af4c3fbc1f606d854c44b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46427 } } }
I20260928 16:42:27.450050 32476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:27.454442 32479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:27.456485 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:27.436462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:52385 (local address 127.0.0.1:46427)
0928 16:42:27.436842 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:27.436850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:27.437125 (+ 275us) server_negotiation.cc:407] Connection header received
0928 16:42:27.437298 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:27.437304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:27.437408 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:27.437533 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:27.439243 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:27.440462 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:27.441871 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:27.442612 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:27.443526 (+ 914us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:27.446406 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:27.446414 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:27.446420 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:27.446470 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:27.451760 (+ 5290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:27.453218 (+ 1458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:27.453225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:27.453230 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:27.453412 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:27.454073 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:27.454079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:27.454086 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:27.454612 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:27.454962 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:27.455443 (+ 481us) server_negotiation.cc:299] Negotiation successful
0928 16:42:27.456031 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 16:42:27.462888 32479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:27.474797 32479 catalog_manager.cc:1347] Generated new cluster ID: 1697b39885594e6192400f78dc5e4444
I20260928 16:42:27.474956 32479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:27.483861 32479 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:27.485803 32479 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:27.499521 32479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d0f7050f7af4c3fbc1f606d854c44b8: Generated new TSK 0
I20260928 16:42:27.501194 32479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:27.509538 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32396
I20260928 16:42:27.511268 32459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:27.613272 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 32396
W20260928 16:42:27.631556 32381 connection.cc:535] client connection to 127.0.0.1:46427 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46427): Connection reset by peer (error 104)
2026-09-28T16:42:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2175 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:27 dist-test-slave-9fn6 krb5kdc[32496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:27 dist-test-slave-9fn6 krb5kdc[32496](info): set up 2 sockets
Sep 28 16:42:27 dist-test-slave-9fn6 krb5kdc[32496](info): commencing operation
krb5kdc: starting...
W20260928 16:42:28.681242 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:28 dist-test-slave-9fn6 krb5kdc[32496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:29.009938 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:29.408022 32509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:29.408408 32509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:29.408556 32509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:29.416409 32509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:29.416586 32509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:29.416688 32509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:29.416810 32509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:29.416877 32509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:29.427896 32509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:29.429678 32509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:29.431453 32509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:29.445399 32514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:29.445399 32515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:29.445399 32517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:29.446267 32509 server_base.cc:1034] running on GCE node
I20260928 16:42:29.447288 32509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:29.449082 32509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:29.450304 32509 hybrid_clock.cc:648] HybridClock initialized: now 1790613749450254 us; error 40 us; skew 500 ppm
I20260928 16:42:29.450695 32509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:29 dist-test-slave-9fn6 krb5kdc[32496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613749, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:29.454263 32509 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:29.455924 32509 webserver.cc:460] Webserver started at http://127.0.0.1:46549/ using document root <none> and password file <none>
I20260928 16:42:29.456761 32509 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:29.456875 32509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:29.457193 32509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:29.460646 32509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "8bb942a3bb4d40f68429633f9b3a6ddb"
format_stamp: "Formatted at 2026-09-28 16:42:29 on dist-test-slave-9fn6"
server_key: "57892295e58690f140ee93a5d4b38785"
server_key_iv: "552d64cc5c761267574dc6eea8c52649"
server_key_version: "encryptionkey@0"
I20260928 16:42:29.461529 32509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "8bb942a3bb4d40f68429633f9b3a6ddb"
format_stamp: "Formatted at 2026-09-28 16:42:29 on dist-test-slave-9fn6"
server_key: "57892295e58690f140ee93a5d4b38785"
server_key_iv: "552d64cc5c761267574dc6eea8c52649"
server_key_version: "encryptionkey@0"
I20260928 16:42:29.467340 32509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:42:29.471195 32524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:29.472779 32509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:42:29.472949 32509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "8bb942a3bb4d40f68429633f9b3a6ddb"
format_stamp: "Formatted at 2026-09-28 16:42:29 on dist-test-slave-9fn6"
server_key: "57892295e58690f140ee93a5d4b38785"
server_key_iv: "552d64cc5c761267574dc6eea8c52649"
server_key_version: "encryptionkey@0"
I20260928 16:42:29.473181 32509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:29.495458 32509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:29.500211 32509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:29.500527 32509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:29.520742 32509 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38201
I20260928 16:42:29.520776 32575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38201 every 8 connection(s)
I20260928 16:42:29.522481 32509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:29.524591 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 32509
I20260928 16:42:29.524859 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:29.525494 11231 external_mini_cluster.cc:1428] Setting key 7da308bfcfacbadb6ac4b98ffe99adaf
I20260928 16:42:29.528353 32576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:29.541615 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:29.527908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:33067 (local address 127.0.0.1:38201)
0928 16:42:29.528857 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:29.528873 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:29.528925 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:42:29.530836 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:29.530865 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:29.531368 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:29.531848 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:29.532547 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:29.534149 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:29.535709 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:29.536530 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:29.536809 (+ 279us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:29.538356 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:29.538365 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:29.538386 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:29.539324 (+ 938us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:29.539449 (+ 125us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:29.539492 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:29.539672 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:29.540022 (+ 350us) server_negotiation.cc:299] Negotiation successful
0928 16:42:29.540277 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":380,"threads_started":1}
I20260928 16:42:29.542438 32576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb: Bootstrap starting.
I20260928 16:42:29.546516 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:29.546639 32576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:29.547819 32576 log.cc:824] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:29.551131 32576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb: No bootstrap required, opened a new log
I20260928 16:42:29.559119 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:29.551515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:57975 (local address 127.0.0.1:38201)
0928 16:42:29.551749 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:29.551762 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:29.552176 (+ 414us) server_negotiation.cc:407] Connection header received
0928 16:42:29.552398 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:29.552403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:29.552477 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:29.552581 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:29.553263 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:29.554156 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:29.556173 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:29.556753 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:29.556866 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:29.558258 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:29.558263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:29.558266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:29.558289 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:29.558340 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:29.558356 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:29.558444 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:29.558665 (+ 221us) server_negotiation.cc:299] Negotiation successful
0928 16:42:29.558732 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 16:42:29.561713 32576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb942a3bb4d40f68429633f9b3a6ddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260928 16:42:29.562201 32576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:29.562310 32576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bb942a3bb4d40f68429633f9b3a6ddb, State: Initialized, Role: FOLLOWER
I20260928 16:42:29.562964 32576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb942a3bb4d40f68429633f9b3a6ddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260928 16:42:29.563222 32576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:29.563359 32576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:29.563541 32576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:29.566135 32576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb942a3bb4d40f68429633f9b3a6ddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260928 16:42:29.566743 32576 leader_election.cc:302] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bb942a3bb4d40f68429633f9b3a6ddb; no voters:
I20260928 16:42:29.567457 32576 leader_election.cc:288] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:29.567587 32587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:29.568269 32587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [term 1 LEADER]: Becoming Leader. State: Replica: 8bb942a3bb4d40f68429633f9b3a6ddb, State: Running, Role: LEADER
I20260928 16:42:29.569018 32587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb942a3bb4d40f68429633f9b3a6ddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260928 16:42:29.570122 32576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:29.572266 32588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bb942a3bb4d40f68429633f9b3a6ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb942a3bb4d40f68429633f9b3a6ddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } } }
I20260928 16:42:29.572763 32588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:29.572705 32589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bb942a3bb4d40f68429633f9b3a6ddb. Latest consensus state: current_term: 1 leader_uuid: "8bb942a3bb4d40f68429633f9b3a6ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb942a3bb4d40f68429633f9b3a6ddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } } }
I20260928 16:42:29.573094 32589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:29.575731 32592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:29.582665 32592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:29.593218 32592 catalog_manager.cc:1347] Generated new cluster ID: 44c086933adc4a0abe3facb408d577ba
I20260928 16:42:29.593366 32592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:29.614141 32592 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:29.616340 32592 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:29.629683 32592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bb942a3bb4d40f68429633f9b3a6ddb: Generated new TSK 0
I20260928 16:42:29.630859 32592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:29.649403 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32509
W20260928 16:42:29.650714 32571 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:33067
W20260928 16:42:29.651576 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32509: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:29.651743 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 32509
W20260928 16:42:29.669191 32494 connection.cc:535] client connection to 127.0.0.1:38201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38201): Connection reset by peer (error 104)
2026-09-28T16:42:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2036 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:29 dist-test-slave-9fn6 krb5kdc[32608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:29 dist-test-slave-9fn6 krb5kdc[32608](info): set up 2 sockets
Sep 28 16:42:29 dist-test-slave-9fn6 krb5kdc[32608](info): commencing operation
krb5kdc: starting...
W20260928 16:42:30.713186 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:30 dist-test-slave-9fn6 krb5kdc[32608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:31.049146 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46409
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:31.453773 32621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:31.454193 32621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:31.454344 32621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:31.462411 32621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:31.462569 32621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:31.462658 32621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:31.462756 32621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:31.462818 32621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:31.474102 32621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:31.475816 32621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:31.477602 32621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:31.491111 32627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:31.491153 32626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:31.491181 32629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:31.491815 32621 server_base.cc:1034] running on GCE node
I20260928 16:42:31.492889 32621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:31.494647 32621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:31.495880 32621 hybrid_clock.cc:648] HybridClock initialized: now 1790613751495830 us; error 41 us; skew 500 ppm
I20260928 16:42:31.496301 32621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:31 dist-test-slave-9fn6 krb5kdc[32608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613751, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:31.499848 32621 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:31.501626 32621 webserver.cc:460] Webserver started at http://127.0.0.1:32925/ using document root <none> and password file <none>
I20260928 16:42:31.502434 32621 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:31.502566 32621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:31.502894 32621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:31.506433 32621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "12664f511ea04b5da003864ffc843d58"
format_stamp: "Formatted at 2026-09-28 16:42:31 on dist-test-slave-9fn6"
server_key: "8c99f55d11e9355a0378b961e681f6f2"
server_key_iv: "0e973cf647f18a693567dab7365eba3a"
server_key_version: "encryptionkey@0"
I20260928 16:42:31.507297 32621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "12664f511ea04b5da003864ffc843d58"
format_stamp: "Formatted at 2026-09-28 16:42:31 on dist-test-slave-9fn6"
server_key: "8c99f55d11e9355a0378b961e681f6f2"
server_key_iv: "0e973cf647f18a693567dab7365eba3a"
server_key_version: "encryptionkey@0"
I20260928 16:42:31.513577 32621 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260928 16:42:31.517735 32636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:31.519338 32621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:42:31.519531 32621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "12664f511ea04b5da003864ffc843d58"
format_stamp: "Formatted at 2026-09-28 16:42:31 on dist-test-slave-9fn6"
server_key: "8c99f55d11e9355a0378b961e681f6f2"
server_key_iv: "0e973cf647f18a693567dab7365eba3a"
server_key_version: "encryptionkey@0"
I20260928 16:42:31.519752 32621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:31.526628 32621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:31.531669 32621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:31.532009 32621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:31.552292 32621 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46409
I20260928 16:42:31.552312 32687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46409 every 8 connection(s)
I20260928 16:42:31.554039 32621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:31.554513 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 32621
I20260928 16:42:31.554682 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:31.555294 11231 external_mini_cluster.cc:1428] Setting key a6b3df773bc31f702952934bccabdcd8
I20260928 16:42:31.559333 32688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:31.564496 32691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:31.557611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:59991 (local address 127.0.0.1:46409)
0928 16:42:31.558478 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:31.558493 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:31.558543 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:42:31.560435 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:31.560463 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:31.560919 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:31.561308 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:31.562599 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:31.562607 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:31.562634 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:31.562672 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:31.562772 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:31.562795 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:31.562990 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:31.563102 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 16:42:31.563334 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":362,"threads_started":1}
I20260928 16:42:31.569475 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:31.573519 32688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58: Bootstrap starting.
I20260928 16:42:31.577939 32691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:31.574844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:56513 (local address 127.0.0.1:46409)
0928 16:42:31.575088 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:31.575097 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:31.575557 (+ 460us) server_negotiation.cc:407] Connection header received
0928 16:42:31.575774 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:31.575778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:31.575844 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:31.575940 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:31.577151 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:31.577156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:31.577159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:31.577177 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:31.577230 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:31.577248 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:31.577364 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:31.577533 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 16:42:31.577610 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 16:42:31.578408 32688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:31.579708 32688 log.cc:824] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:31.583261 32688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58: No bootstrap required, opened a new log
I20260928 16:42:31.591739 32688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12664f511ea04b5da003864ffc843d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } }
I20260928 16:42:31.592294 32688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:31.592407 32688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12664f511ea04b5da003864ffc843d58, State: Initialized, Role: FOLLOWER
I20260928 16:42:31.593173 32688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12664f511ea04b5da003864ffc843d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } }
I20260928 16:42:31.593451 32688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:31.593600 32688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:31.593818 32688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:31.596451 32688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12664f511ea04b5da003864ffc843d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } }
I20260928 16:42:31.597208 32688 leader_election.cc:302] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12664f511ea04b5da003864ffc843d58; no voters:
I20260928 16:42:31.597913 32688 leader_election.cc:288] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:31.598067 32698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:31.598886 32698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [term 1 LEADER]: Becoming Leader. State: Replica: 12664f511ea04b5da003864ffc843d58, State: Running, Role: LEADER
I20260928 16:42:31.599519 32698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12664f511ea04b5da003864ffc843d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } }
I20260928 16:42:31.600383 32688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:31.602975 32699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12664f511ea04b5da003864ffc843d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12664f511ea04b5da003864ffc843d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } } }
I20260928 16:42:31.603565 32700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12664f511ea04b5da003864ffc843d58. Latest consensus state: current_term: 1 leader_uuid: "12664f511ea04b5da003864ffc843d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12664f511ea04b5da003864ffc843d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } } }
I20260928 16:42:31.604128 32700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:31.604303 32699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:31.613754 32707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:31.620276 32707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:31.631330 32707 catalog_manager.cc:1347] Generated new cluster ID: 06303c8304e344148713336d18e6790e
I20260928 16:42:31.631467 32707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:31.650499 32707 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:31.651237 32707 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:31.673233 32707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12664f511ea04b5da003864ffc843d58: Generated new TSK 0
I20260928 16:42:31.674434 32707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:31.680141 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32621
W20260928 16:42:31.681496 32683 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:59991
W20260928 16:42:31.682276 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32621: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:31.682411 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 32621
2026-09-28T16:42:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:31 dist-test-slave-9fn6 krb5kdc[32719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:31 dist-test-slave-9fn6 krb5kdc[32719](info): set up 2 sockets
Sep 28 16:42:31 dist-test-slave-9fn6 krb5kdc[32719](info): commencing operation
krb5kdc: starting...
W20260928 16:42:32.753253 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:32 dist-test-slave-9fn6 krb5kdc[32719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:33.083973 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:33.480405 32732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:33.480834 32732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:33.480939 32732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:33.489154 32732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:33.489305 32732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:33.489353 32732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:33.489424 32732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:33.489470 32732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:33.500705 32732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:33.502452 32732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:33.504063 32732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:33.518517 32738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:33.518582 32740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:33.518407 32737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:33.519574 32732 server_base.cc:1034] running on GCE node
I20260928 16:42:33.520761 32732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:33.522776 32732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:33.524065 32732 hybrid_clock.cc:648] HybridClock initialized: now 1790613753523962 us; error 79 us; skew 500 ppm
I20260928 16:42:33.524518 32732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:33 dist-test-slave-9fn6 krb5kdc[32719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613753, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:33.528502 32732 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:33.530586 32732 webserver.cc:460] Webserver started at http://127.0.0.1:37557/ using document root <none> and password file <none>
I20260928 16:42:33.531494 32732 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:33.531620 32732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:33.531961 32732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:33.535930 32732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "03fb074d027d417ab778d9eb05f97b30"
format_stamp: "Formatted at 2026-09-28 16:42:33 on dist-test-slave-9fn6"
server_key: "3fb3835ebdfc06c6fa9f00b82c709a50"
server_key_iv: "95110b732ca32f7a49c1815e5868f6f0"
server_key_version: "encryptionkey@0"
I20260928 16:42:33.536957 32732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "03fb074d027d417ab778d9eb05f97b30"
format_stamp: "Formatted at 2026-09-28 16:42:33 on dist-test-slave-9fn6"
server_key: "3fb3835ebdfc06c6fa9f00b82c709a50"
server_key_iv: "95110b732ca32f7a49c1815e5868f6f0"
server_key_version: "encryptionkey@0"
I20260928 16:42:33.543649 32732 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260928 16:42:33.547848 32747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:33.549551 32732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:42:33.549753 32732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "03fb074d027d417ab778d9eb05f97b30"
format_stamp: "Formatted at 2026-09-28 16:42:33 on dist-test-slave-9fn6"
server_key: "3fb3835ebdfc06c6fa9f00b82c709a50"
server_key_iv: "95110b732ca32f7a49c1815e5868f6f0"
server_key_version: "encryptionkey@0"
I20260928 16:42:33.549984 32732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:33.575510 32732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:33.580726 32732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:33.581040 32732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:33.601385 32732 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35195
I20260928 16:42:33.601409 331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35195 every 8 connection(s)
I20260928 16:42:33.603060 32732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:33.607617 332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:33.610235 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 32732
I20260928 16:42:33.610426 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:33.610915 11231 external_mini_cluster.cc:1428] Setting key 1599a97497d62cecd0b52a92065ab07a
I20260928 16:42:33.621681 332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30: Bootstrap starting.
I20260928 16:42:33.625989 332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30: Neither blocks nor log segments found. Creating new log.
Sep 28 16:42:33 dist-test-slave-9fn6 krb5kdc[32719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 16:42:33.627238 332 log.cc:824] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:33.631381 332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30: No bootstrap required, opened a new log
I20260928 16:42:33.637696 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:33.613148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:43723 (local address 127.0.0.1:35195)
0928 16:42:33.614062 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:33.614076 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:33.614128 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:42:33.616000 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:33.616028 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:33.616521 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:33.617001 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:33.618755 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:33.620933 (+ 2178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:33.622393 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:33.623459 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:33.623695 (+ 236us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:33.628150 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:33.628174 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:33.628193 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:33.628239 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:33.632151 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:33.633105 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:33.633115 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:33.633124 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:33.633393 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:33.633847 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:33.633852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:33.633856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:33.634710 (+ 854us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:33.635072 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:33.635591 (+ 519us) server_negotiation.cc:299] Negotiation successful
0928 16:42:33.636000 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":383,"threads_started":1}
I20260928 16:42:33.642540 332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03fb074d027d417ab778d9eb05f97b30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35195 } }
I20260928 16:42:33.643100 332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:33.643180 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:33.643225 332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb074d027d417ab778d9eb05f97b30, State: Initialized, Role: FOLLOWER
I20260928 16:42:33.643946 332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03fb074d027d417ab778d9eb05f97b30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35195 } }
I20260928 16:42:33.644207 332 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:33.644364 332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:33.644539 332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:33.647351 332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03fb074d027d417ab778d9eb05f97b30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35195 } }
I20260928 16:42:33.648033 332 leader_election.cc:302] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03fb074d027d417ab778d9eb05f97b30; no voters:
I20260928 16:42:33.648891 332 leader_election.cc:288] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:33.649078 341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:33.651342 341 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [term 1 LEADER]: Becoming Leader. State: Replica: 03fb074d027d417ab778d9eb05f97b30, State: Running, Role: LEADER
I20260928 16:42:33.651741 332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:33.652041 341 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03fb074d027d417ab778d9eb05f97b30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35195 } }
I20260928 16:42:33.660256 344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03fb074d027d417ab778d9eb05f97b30. Latest consensus state: current_term: 1 leader_uuid: "03fb074d027d417ab778d9eb05f97b30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03fb074d027d417ab778d9eb05f97b30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35195 } } }
I20260928 16:42:33.660677 344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:33.659327 343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03fb074d027d417ab778d9eb05f97b30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03fb074d027d417ab778d9eb05f97b30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35195 } } }
I20260928 16:42:33.661303 343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:33.664222 350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:33.671481 350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:33.671952 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:33.648985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:36275 (local address 127.0.0.1:35195)
0928 16:42:33.649276 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:33.649283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:33.650772 (+ 1489us) server_negotiation.cc:407] Connection header received
0928 16:42:33.651106 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:33.651112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:33.651232 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:33.651368 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:33.653879 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:33.655330 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:33.657878 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:33.658587 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:33.658740 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:33.662730 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:33.662739 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:33.662745 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:33.662807 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:33.667015 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:33.668231 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:33.668236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:33.668240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:33.668451 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:33.669195 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:33.669201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:33.669204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:33.669692 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:33.670004 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:33.671324 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0928 16:42:33.671565 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260928 16:42:33.683931 350 catalog_manager.cc:1347] Generated new cluster ID: b0f3aeb728f8409f8911b3f320126869
I20260928 16:42:33.684092 350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:33.701097 350 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:33.703413 350 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:33.734203 350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03fb074d027d417ab778d9eb05f97b30: Generated new TSK 0
I20260928 16:42:33.735404 350 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:33.748479 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 32732
I20260928 16:42:33.749933 327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:33.854647 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 32732
W20260928 16:42:33.873255 32717 connection.cc:535] client connection to 127.0.0.1:35195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35195): Connection reset by peer (error 104)
2026-09-28T16:42:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2172 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:33 dist-test-slave-9fn6 krb5kdc[365](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:33 dist-test-slave-9fn6 krb5kdc[365](info): set up 2 sockets
Sep 28 16:42:33 dist-test-slave-9fn6 krb5kdc[365](info): commencing operation
krb5kdc: starting...
W20260928 16:42:34.925364 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:35 dist-test-slave-9fn6 krb5kdc[365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:35.256911 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:35.650616 378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:35.651014 378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:35.651152 378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:35.659231 378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:35.659396 378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:35.659456 378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:35.659557 378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:35.659615 378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:35.671000 378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32921
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:35.672855 378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:35.674564 378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:35.688490 383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:35.688553 386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:35.688530 384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:35.689561 378 server_base.cc:1034] running on GCE node
I20260928 16:42:35.690630 378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:35.692570 378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:35.693848 378 hybrid_clock.cc:648] HybridClock initialized: now 1790613755693784 us; error 60 us; skew 500 ppm
I20260928 16:42:35.694285 378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:35 dist-test-slave-9fn6 krb5kdc[365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613755, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:35.702363 378 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:35.704571 378 webserver.cc:460] Webserver started at http://127.0.0.1:38653/ using document root <none> and password file <none>
I20260928 16:42:35.705612 378 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:35.705766 378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:35.706147 378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:35.710157 378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "12195f70ddb1470cb1416f059516be05"
format_stamp: "Formatted at 2026-09-28 16:42:35 on dist-test-slave-9fn6"
server_key: "0af6c428ad4cb9a9ce91fae168b54160"
server_key_iv: "a04760501d559c48337983ce82ddf64e"
server_key_version: "encryptionkey@0"
I20260928 16:42:35.711110 378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "12195f70ddb1470cb1416f059516be05"
format_stamp: "Formatted at 2026-09-28 16:42:35 on dist-test-slave-9fn6"
server_key: "0af6c428ad4cb9a9ce91fae168b54160"
server_key_iv: "a04760501d559c48337983ce82ddf64e"
server_key_version: "encryptionkey@0"
I20260928 16:42:35.717698 378 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260928 16:42:35.722033 393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:35.723798 378 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260928 16:42:35.723999 378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "12195f70ddb1470cb1416f059516be05"
format_stamp: "Formatted at 2026-09-28 16:42:35 on dist-test-slave-9fn6"
server_key: "0af6c428ad4cb9a9ce91fae168b54160"
server_key_iv: "a04760501d559c48337983ce82ddf64e"
server_key_version: "encryptionkey@0"
I20260928 16:42:35.724225 378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:35.745280 378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:35.750394 378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:35.750720 378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:35.770298 378 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32921
I20260928 16:42:35.770319 444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32921 every 8 connection(s)
I20260928 16:42:35.771945 378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:35.775956 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 378
I20260928 16:42:35.776177 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:35.776628 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:35.776710 11231 external_mini_cluster.cc:1428] Setting key 20dcee0287669383e4bbd0cb429f6b4a
I20260928 16:42:35.792421 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:35.778747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:34513 (local address 127.0.0.1:32921)
0928 16:42:35.779625 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:35.779639 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:35.779700 (+ 61us) server_negotiation.cc:407] Connection header received
0928 16:42:35.781988 (+ 2288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:35.782021 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:35.782583 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:35.783037 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:35.783808 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:35.785378 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:35.787211 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:35.788084 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:35.788227 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:35.789682 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:35.789691 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:35.789709 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:35.790411 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:35.790512 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:35.790540 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:35.790676 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:35.790937 (+ 261us) server_negotiation.cc:299] Negotiation successful
0928 16:42:35.791153 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":389,"threads_started":1}
I20260928 16:42:35.792542 445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05: Bootstrap starting.
I20260928 16:42:35.796782 445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:35.796962 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:35.798013 445 log.cc:824] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:35.801700 445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05: No bootstrap required, opened a new log
I20260928 16:42:35.809607 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:35.802010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:41849 (local address 127.0.0.1:32921)
0928 16:42:35.802228 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:35.802235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:35.802828 (+ 593us) server_negotiation.cc:407] Connection header received
0928 16:42:35.803037 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:35.803041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:35.803114 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:35.803209 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:35.803928 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:35.804833 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:35.806771 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:35.807271 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:35.807376 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:35.808814 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:35.808819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:35.808823 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:35.808848 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:35.808894 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:35.808911 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:35.808999 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:35.809171 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 16:42:35.809226 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:42:35.812253 445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12195f70ddb1470cb1416f059516be05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } }
I20260928 16:42:35.812765 445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:35.812878 445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12195f70ddb1470cb1416f059516be05, State: Initialized, Role: FOLLOWER
I20260928 16:42:35.813550 445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12195f70ddb1470cb1416f059516be05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } }
I20260928 16:42:35.813807 445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:35.813953 445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:35.814126 445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:35.816534 445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12195f70ddb1470cb1416f059516be05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } }
I20260928 16:42:35.817209 445 leader_election.cc:302] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12195f70ddb1470cb1416f059516be05; no voters:
I20260928 16:42:35.817924 445 leader_election.cc:288] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:35.818051 455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:35.818681 455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [term 1 LEADER]: Becoming Leader. State: Replica: 12195f70ddb1470cb1416f059516be05, State: Running, Role: LEADER
I20260928 16:42:35.819357 455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12195f70ddb1470cb1416f059516be05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } }
I20260928 16:42:35.820578 445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:35.822649 456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12195f70ddb1470cb1416f059516be05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12195f70ddb1470cb1416f059516be05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } } }
I20260928 16:42:35.823163 456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:35.822732 457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12195f70ddb1470cb1416f059516be05. Latest consensus state: current_term: 1 leader_uuid: "12195f70ddb1470cb1416f059516be05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12195f70ddb1470cb1416f059516be05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } } }
I20260928 16:42:35.824908 457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:35.825158 462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:35.831557 462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:35.841782 462 catalog_manager.cc:1347] Generated new cluster ID: 0bab71970071436ea9f4c983761bdd60
I20260928 16:42:35.841924 462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:35.872493 462 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:35.874084 462 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:35.883337 462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12195f70ddb1470cb1416f059516be05: Generated new TSK 0
I20260928 16:42:35.884390 462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:35.896238 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 378
W20260928 16:42:35.898102 440 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:34513
W20260928 16:42:35.899042 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 378: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:35.899210 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 378
W20260928 16:42:35.916448 363 connection.cc:535] client connection to 127.0.0.1:32921 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32921): Connection reset by peer (error 104)
2026-09-28T16:42:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2044 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:36 dist-test-slave-9fn6 krb5kdc[476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:36 dist-test-slave-9fn6 krb5kdc[476](info): set up 2 sockets
Sep 28 16:42:36 dist-test-slave-9fn6 krb5kdc[476](info): commencing operation
krb5kdc: starting...
W20260928 16:42:36.961397 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.008s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:37 dist-test-slave-9fn6 krb5kdc[476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:42:37.289742 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:37.681706 489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:37.682137 489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:37.682301 489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:37.690439 489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:37.690629 489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:37.690721 489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:37.690809 489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:37.690873 489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:37.702378 489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:37.704265 489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:37.706029 489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:37.720217 494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:37.720156 497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:37.720124 495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:37.721114 489 server_base.cc:1034] running on GCE node
I20260928 16:42:37.722100 489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:37.724066 489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:37.725301 489 hybrid_clock.cc:648] HybridClock initialized: now 1790613757725243 us; error 58 us; skew 500 ppm
I20260928 16:42:37.725742 489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:37 dist-test-slave-9fn6 krb5kdc[476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613757, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:37.729493 489 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:42:37.731364 489 webserver.cc:460] Webserver started at http://127.0.0.1:40503/ using document root <none> and password file <none>
I20260928 16:42:37.732239 489 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:37.732360 489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:37.732736 489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:37.736542 489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "2a2ccd79bdd943508bd95324a7081727"
format_stamp: "Formatted at 2026-09-28 16:42:37 on dist-test-slave-9fn6"
server_key: "77a4989d9324de17a87218dd020e1b49"
server_key_iv: "ff1c65d587b0f0bdd5594497f0b41427"
server_key_version: "encryptionkey@0"
I20260928 16:42:37.737504 489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "2a2ccd79bdd943508bd95324a7081727"
format_stamp: "Formatted at 2026-09-28 16:42:37 on dist-test-slave-9fn6"
server_key: "77a4989d9324de17a87218dd020e1b49"
server_key_iv: "ff1c65d587b0f0bdd5594497f0b41427"
server_key_version: "encryptionkey@0"
I20260928 16:42:37.744081 489 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:42:37.748383 504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:37.750064 489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:42:37.750257 489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "2a2ccd79bdd943508bd95324a7081727"
format_stamp: "Formatted at 2026-09-28 16:42:37 on dist-test-slave-9fn6"
server_key: "77a4989d9324de17a87218dd020e1b49"
server_key_iv: "ff1c65d587b0f0bdd5594497f0b41427"
server_key_version: "encryptionkey@0"
I20260928 16:42:37.750475 489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:37.757527 489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:37.762570 489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:37.762910 489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:37.783033 489 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40807
I20260928 16:42:37.783051 555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40807 every 8 connection(s)
I20260928 16:42:37.784855 489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:37.789641 556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:37.794580 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 489
I20260928 16:42:37.794770 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:37.795271 11231 external_mini_cluster.cc:1428] Setting key 5d8eb2b7b90ef43d825832f728243163
I20260928 16:42:37.805019 556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727: Bootstrap starting.
I20260928 16:42:37.805101 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:37.797547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:44449 (local address 127.0.0.1:40807)
0928 16:42:37.798467 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:37.798483 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:37.798536 (+ 53us) server_negotiation.cc:407] Connection header received
0928 16:42:37.800448 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:37.800482 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:37.801061 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:37.801523 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:37.802931 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:37.802940 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:37.802956 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:37.802976 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:37.803068 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:37.803099 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:37.803337 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:37.803456 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 16:42:37.803750 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":386,"threads_started":1}
I20260928 16:42:37.809281 556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:37.809741 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:37.810567 556 log.cc:824] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:37.814278 556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727: No bootstrap required, opened a new log
I20260928 16:42:37.817467 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:37.814641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:42953 (local address 127.0.0.1:40807)
0928 16:42:37.814876 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:37.814885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:37.815339 (+ 454us) server_negotiation.cc:407] Connection header received
0928 16:42:37.815549 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:37.815553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:37.815612 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:37.815704 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:37.816765 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:37.816770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:37.816773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:37.816792 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:37.816840 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:37.816860 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:37.816962 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:37.817075 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 16:42:37.817134 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:42:37.824321 556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2ccd79bdd943508bd95324a7081727" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } }
I20260928 16:42:37.824862 556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:37.824968 556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2ccd79bdd943508bd95324a7081727, State: Initialized, Role: FOLLOWER
I20260928 16:42:37.825640 556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2ccd79bdd943508bd95324a7081727" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } }
I20260928 16:42:37.825901 556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:37.826036 556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:37.826215 556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:37.829113 556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2ccd79bdd943508bd95324a7081727" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } }
I20260928 16:42:37.829695 556 leader_election.cc:302] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a2ccd79bdd943508bd95324a7081727; no voters:
I20260928 16:42:37.830390 556 leader_election.cc:288] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:37.830516 566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:37.831847 566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [term 1 LEADER]: Becoming Leader. State: Replica: 2a2ccd79bdd943508bd95324a7081727, State: Running, Role: LEADER
I20260928 16:42:37.832480 566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2ccd79bdd943508bd95324a7081727" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } }
I20260928 16:42:37.833331 556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:37.835670 568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a2ccd79bdd943508bd95324a7081727. Latest consensus state: current_term: 1 leader_uuid: "2a2ccd79bdd943508bd95324a7081727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2ccd79bdd943508bd95324a7081727" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } } }
I20260928 16:42:37.835620 567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a2ccd79bdd943508bd95324a7081727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2ccd79bdd943508bd95324a7081727" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } } }
I20260928 16:42:37.836126 568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:37.836125 567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:37.838351 574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:37.844841 574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:37.856187 574 catalog_manager.cc:1347] Generated new cluster ID: f611222cecd743b19e7a0ff2b0729c23
I20260928 16:42:37.856356 574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:37.873579 574 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:37.874146 574 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:37.883074 574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a2ccd79bdd943508bd95324a7081727: Generated new TSK 0
I20260928 16:42:37.884493 574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:37.896225 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 489
W20260928 16:42:37.898010 551 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:44449
W20260928 16:42:37.898917 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 489: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:37.899107 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 489
2026-09-28T16:42:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2000 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:38 dist-test-slave-9fn6 krb5kdc[587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:38 dist-test-slave-9fn6 krb5kdc[587](info): set up 2 sockets
Sep 28 16:42:38 dist-test-slave-9fn6 krb5kdc[587](info): commencing operation
krb5kdc: starting...
W20260928 16:42:38.953351 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.008s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:39 dist-test-slave-9fn6 krb5kdc[587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:42:39.280246 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:33781
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:33781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:39.677074 600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:39.677464 600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:39.677598 600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:39.685622 600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:39.685770 600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:39.685834 600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:39.685918 600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:39.685983 600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:39.697414 600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:33781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:33781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:39.699151 600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:39.700883 600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:39.714664 608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:39.714670 605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:39.715307 606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:39.716517 600 server_base.cc:1034] running on GCE node
I20260928 16:42:39.717583 600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:39.719524 600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:39.720711 600 hybrid_clock.cc:648] HybridClock initialized: now 1790613759720660 us; error 48 us; skew 500 ppm
I20260928 16:42:39.721127 600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:39 dist-test-slave-9fn6 krb5kdc[587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613759, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:39.724718 600 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:42:39.726512 600 webserver.cc:460] Webserver started at http://127.10.247.254:35173/ using document root <none> and password file <none>
I20260928 16:42:39.727394 600 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:39.727525 600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:39.727885 600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:39.731765 600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "e4d6aa3bc9584c64907585c1c29d6cf0"
format_stamp: "Formatted at 2026-09-28 16:42:39 on dist-test-slave-9fn6"
server_key: "d3405ecb924ed3fadc6f8e716f585505"
server_key_iv: "afc4b6a64cca1405ae8608a0b96962e3"
server_key_version: "encryptionkey@0"
I20260928 16:42:39.732779 600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "e4d6aa3bc9584c64907585c1c29d6cf0"
format_stamp: "Formatted at 2026-09-28 16:42:39 on dist-test-slave-9fn6"
server_key: "d3405ecb924ed3fadc6f8e716f585505"
server_key_iv: "afc4b6a64cca1405ae8608a0b96962e3"
server_key_version: "encryptionkey@0"
I20260928 16:42:39.739125 600 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260928 16:42:39.743168 615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:39.744832 600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:42:39.745024 600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "e4d6aa3bc9584c64907585c1c29d6cf0"
format_stamp: "Formatted at 2026-09-28 16:42:39 on dist-test-slave-9fn6"
server_key: "d3405ecb924ed3fadc6f8e716f585505"
server_key_iv: "afc4b6a64cca1405ae8608a0b96962e3"
server_key_version: "encryptionkey@0"
I20260928 16:42:39.745239 600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:39.770207 600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:39.774755 600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:39.775019 600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:39.794184 600 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:33781
I20260928 16:42:39.794196 666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:33781 every 8 connection(s)
I20260928 16:42:39.795799 600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:39.796285 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 600
I20260928 16:42:39.796453 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:39.797277 11231 external_mini_cluster.cc:1428] Setting key f96a74e1b864f9d0f645a45b45727f2f
I20260928 16:42:39.801168 667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 16:42:39 dist-test-slave-9fn6 krb5kdc[587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:39.813468 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0: Bootstrap starting.
I20260928 16:42:39.817641 667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:39.818888 667 log.cc:824] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:39.821281 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:39.799425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50939 (local address 127.10.247.254:33781)
0928 16:42:39.800267 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:39.800280 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:39.800327 (+ 47us) server_negotiation.cc:407] Connection header received
0928 16:42:39.802258 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:39.802286 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:39.802733 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:39.803128 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:39.804828 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:39.806306 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:39.807683 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:39.808453 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:39.808656 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:39.813249 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:39.813260 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:39.813279 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:39.813331 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:39.816542 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:39.817435 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:39.817443 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:39.817451 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:39.817647 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:39.818154 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:39.818158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:39.818160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:39.818760 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:39.819099 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:39.819608 (+ 509us) server_negotiation.cc:299] Negotiation successful
0928 16:42:39.820003 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":343,"threads_started":1}
I20260928 16:42:39.822449 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0: No bootstrap required, opened a new log
I20260928 16:42:39.825583 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:39.832828 667 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d6aa3bc9584c64907585c1c29d6cf0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33781 } }
I20260928 16:42:39.833429 667 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:39.833559 667 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4d6aa3bc9584c64907585c1c29d6cf0, State: Initialized, Role: FOLLOWER
I20260928 16:42:39.834304 667 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d6aa3bc9584c64907585c1c29d6cf0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33781 } }
I20260928 16:42:39.834555 667 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:39.834724 667 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:39.834933 667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:39.838284 667 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d6aa3bc9584c64907585c1c29d6cf0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33781 } }
I20260928 16:42:39.839030 667 leader_election.cc:302] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4d6aa3bc9584c64907585c1c29d6cf0; no voters:
I20260928 16:42:39.839937 667 leader_election.cc:288] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:39.840049 677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:39.840948 677 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [term 1 LEADER]: Becoming Leader. State: Replica: e4d6aa3bc9584c64907585c1c29d6cf0, State: Running, Role: LEADER
I20260928 16:42:39.841737 677 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d6aa3bc9584c64907585c1c29d6cf0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33781 } }
I20260928 16:42:39.844363 677 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4d6aa3bc9584c64907585c1c29d6cf0. Latest consensus state: current_term: 1 leader_uuid: "e4d6aa3bc9584c64907585c1c29d6cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d6aa3bc9584c64907585c1c29d6cf0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33781 } } }
I20260928 16:42:39.844791 677 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:39.848039 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:39.830456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56933 (local address 127.10.247.254:33781)
0928 16:42:39.830833 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:39.830845 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:39.831071 (+ 226us) server_negotiation.cc:407] Connection header received
0928 16:42:39.831273 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:39.831277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:39.831347 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:39.831434 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:39.833624 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:39.834502 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:39.836255 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:39.836787 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:39.836893 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:39.840098 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:39.840105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:39.840110 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:39.840198 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:39.843494 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:39.844531 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:39.844537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:39.844540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:39.844794 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:39.845449 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:39.845454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:39.845457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:39.845899 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:39.846201 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:39.847448 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0928 16:42:39.847658 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 16:42:39.849499 667 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:39.850219 682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:39.850425 679 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4d6aa3bc9584c64907585c1c29d6cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d6aa3bc9584c64907585c1c29d6cf0" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 33781 } } }
I20260928 16:42:39.850795 679 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:39.855307 682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:39.867141 682 catalog_manager.cc:1347] Generated new cluster ID: a17e444900ad497e91f861f42c69ecee
I20260928 16:42:39.867309 682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:39.878800 682 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:39.880275 682 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:39.886672 682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4d6aa3bc9584c64907585c1c29d6cf0: Generated new TSK 0
I20260928 16:42:39.887708 682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:39.894562 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 600
I20260928 16:42:39.896804 662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:39.998085 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 600
W20260928 16:42:40.016510 585 connection.cc:535] client connection to 127.10.247.254:33781 recv error: Network error: failed to read from TLS socket (remote: 127.10.247.254:33781): Connection reset by peer (error 104)
2026-09-28T16:42:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2098 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:40 dist-test-slave-9fn6 krb5kdc[700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:40 dist-test-slave-9fn6 krb5kdc[700](info): set up 2 sockets
Sep 28 16:42:40 dist-test-slave-9fn6 krb5kdc[700](info): commencing operation
krb5kdc: starting...
W20260928 16:42:41.065357 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.007s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:41 dist-test-slave-9fn6 krb5kdc[700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:42:41.396067 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44243
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:44243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:41.785132 713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:41.785534 713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:41.785676 713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:41.793612 713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:41.793740 713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:41.793818 713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:41.793892 713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:41.793937 713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:41.805158 713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:44243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:41.806879 713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:41.808566 713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:41.822146 718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:41.822168 719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:41.822280 721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:41.822804 713 server_base.cc:1034] running on GCE node
I20260928 16:42:41.823891 713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:41.825614 713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:41.826835 713 hybrid_clock.cc:648] HybridClock initialized: now 1790613761826787 us; error 42 us; skew 500 ppm
I20260928 16:42:41.827232 713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:41 dist-test-slave-9fn6 krb5kdc[700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613761, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:41.835165 713 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:42:41.838346 713 webserver.cc:460] Webserver started at http://127.10.247.254:37041/ using document root <none> and password file <none>
I20260928 16:42:41.839336 713 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:41.839454 713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:41.839766 713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:41.843480 713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "68c834ae2525415380e73f2c06d74cd2"
format_stamp: "Formatted at 2026-09-28 16:42:41 on dist-test-slave-9fn6"
server_key: "ada463a335b27a80fa4980d11c590de6"
server_key_iv: "ad6128bd5f56d56715623484f38907a8"
server_key_version: "encryptionkey@0"
I20260928 16:42:41.844345 713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "68c834ae2525415380e73f2c06d74cd2"
format_stamp: "Formatted at 2026-09-28 16:42:41 on dist-test-slave-9fn6"
server_key: "ada463a335b27a80fa4980d11c590de6"
server_key_iv: "ad6128bd5f56d56715623484f38907a8"
server_key_version: "encryptionkey@0"
I20260928 16:42:41.850473 713 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260928 16:42:41.854419 728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:41.856010 713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:42:41.856199 713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "68c834ae2525415380e73f2c06d74cd2"
format_stamp: "Formatted at 2026-09-28 16:42:41 on dist-test-slave-9fn6"
server_key: "ada463a335b27a80fa4980d11c590de6"
server_key_iv: "ad6128bd5f56d56715623484f38907a8"
server_key_version: "encryptionkey@0"
I20260928 16:42:41.856412 713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:41.874267 713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:41.878834 713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:41.879148 713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:41.899806 713 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:44243
I20260928 16:42:41.899812 781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:44243 every 8 connection(s)
I20260928 16:42:41.901583 713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:41.906225 782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:41.910746 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 713
I20260928 16:42:41.910966 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:41.911610 11231 external_mini_cluster.cc:1428] Setting key 878e49891f9850aad063aafb367327cc
I20260928 16:42:41.921171 782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2: Bootstrap starting.
I20260928 16:42:41.925562 782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:41.926887 782 log.cc:824] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:41.927103 785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:41.913949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55795 (local address 127.10.247.254:44243)
0928 16:42:41.914837 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:41.914850 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:41.914900 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:42:41.916825 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:41.916856 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:41.917413 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:41.917825 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:41.918545 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:41.919989 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:41.921879 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:41.922618 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:41.922779 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:41.924288 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:41.924296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:41.924313 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:41.925000 (+ 687us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:41.925094 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:41.925132 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:41.925257 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:41.925618 (+ 361us) server_negotiation.cc:299] Negotiation successful
0928 16:42:41.925841 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":400,"threads_started":1}
I20260928 16:42:41.930920 782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2: No bootstrap required, opened a new log
I20260928 16:42:41.932564 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:41.942039 782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c834ae2525415380e73f2c06d74cd2" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44243 } }
I20260928 16:42:41.942557 782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:41.942682 782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c834ae2525415380e73f2c06d74cd2, State: Initialized, Role: FOLLOWER
I20260928 16:42:41.943415 782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c834ae2525415380e73f2c06d74cd2" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44243 } }
I20260928 16:42:41.943703 782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:41.943868 782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:41.944039 782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:41.945801 785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:41.938854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39577 (local address 127.10.247.254:44243)
0928 16:42:41.939124 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:41.939132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:41.939519 (+ 387us) server_negotiation.cc:407] Connection header received
0928 16:42:41.939743 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:41.939753 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:41.939838 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:41.939936 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:41.940563 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:41.941462 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:41.943020 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:41.943510 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:41.943637 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:41.944955 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:41.944960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:41.944964 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:41.944986 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:41.945073 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:41.945094 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:41.945202 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:41.945405 (+ 203us) server_negotiation.cc:299] Negotiation successful
0928 16:42:41.945479 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:42:41.947228 782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c834ae2525415380e73f2c06d74cd2" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44243 } }
I20260928 16:42:41.947873 782 leader_election.cc:302] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68c834ae2525415380e73f2c06d74cd2; no voters:
I20260928 16:42:41.948499 782 leader_election.cc:288] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:41.948700 792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:41.949445 792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [term 1 LEADER]: Becoming Leader. State: Replica: 68c834ae2525415380e73f2c06d74cd2, State: Running, Role: LEADER
I20260928 16:42:41.950057 792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c834ae2525415380e73f2c06d74cd2" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44243 } }
I20260928 16:42:41.951277 782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:41.953192 793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68c834ae2525415380e73f2c06d74cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c834ae2525415380e73f2c06d74cd2" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44243 } } }
I20260928 16:42:41.953220 794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c834ae2525415380e73f2c06d74cd2. Latest consensus state: current_term: 1 leader_uuid: "68c834ae2525415380e73f2c06d74cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c834ae2525415380e73f2c06d74cd2" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44243 } } }
I20260928 16:42:41.953691 793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:41.953836 794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:41.955451 797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:41.961771 797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:41.972792 797 catalog_manager.cc:1347] Generated new cluster ID: 860e69632ef44a02b01fe4510fae6df7
I20260928 16:42:41.972947 797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:41.993234 797 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:41.995620 797 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:42.006916 797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68c834ae2525415380e73f2c06d74cd2: Generated new TSK 0
I20260928 16:42:42.008387 797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:42.025537 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 713
W20260928 16:42:42.026906 777 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55795
W20260928 16:42:42.027837 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 713: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:42.027990 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 713
W20260928 16:42:42.045846 698 connection.cc:535] client connection to 127.10.247.254:44243 recv error: Network error: failed to read from TLS socket (remote: 127.10.247.254:44243): Connection reset by peer (error 104)
2026-09-28T16:42:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2029 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:42 dist-test-slave-9fn6 krb5kdc[813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:42 dist-test-slave-9fn6 krb5kdc[813](info): set up 2 sockets
Sep 28 16:42:42 dist-test-slave-9fn6 krb5kdc[813](info): commencing operation
krb5kdc: starting...
W20260928 16:42:43.081182 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:43 dist-test-slave-9fn6 krb5kdc[813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:42:43.407781 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:43765
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:32895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:43765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:43.798559 826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:43.798985 826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:43.799158 826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:43.807289 826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:43.807454 826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:43.807540 826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:43.807631 826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:43.807701 826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:43.818871 826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:32895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:43765
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:43765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:43.820736 826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:43.822451 826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:43.835630 831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:43.835716 832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:43.835739 834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:43.836522 826 server_base.cc:1034] running on GCE node
I20260928 16:42:43.837540 826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:43.839356 826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:43.840559 826 hybrid_clock.cc:648] HybridClock initialized: now 1790613763840504 us; error 50 us; skew 500 ppm
I20260928 16:42:43.840999 826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:43 dist-test-slave-9fn6 krb5kdc[813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613763, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:43.844478 826 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:42:43.846238 826 webserver.cc:460] Webserver started at http://127.10.247.254:34243/ using document root <none> and password file <none>
I20260928 16:42:43.847064 826 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:43.847173 826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:43.847470 826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:43.851126 826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "d50c6686374c4590a356c2312a4a8d84"
format_stamp: "Formatted at 2026-09-28 16:42:43 on dist-test-slave-9fn6"
server_key: "2784b89842ad8b24f8f4b5957ee8e011"
server_key_iv: "1e6f0f555cf708baa86b430bd6ec4ea8"
server_key_version: "encryptionkey@0"
I20260928 16:42:43.851969 826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "d50c6686374c4590a356c2312a4a8d84"
format_stamp: "Formatted at 2026-09-28 16:42:43 on dist-test-slave-9fn6"
server_key: "2784b89842ad8b24f8f4b5957ee8e011"
server_key_iv: "1e6f0f555cf708baa86b430bd6ec4ea8"
server_key_version: "encryptionkey@0"
I20260928 16:42:43.858224 826 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260928 16:42:43.862622 841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:43.864377 826 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260928 16:42:43.864599 826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "d50c6686374c4590a356c2312a4a8d84"
format_stamp: "Formatted at 2026-09-28 16:42:43 on dist-test-slave-9fn6"
server_key: "2784b89842ad8b24f8f4b5957ee8e011"
server_key_iv: "1e6f0f555cf708baa86b430bd6ec4ea8"
server_key_version: "encryptionkey@0"
I20260928 16:42:43.864886 826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:43.871865 826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:43.876765 826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:43.877092 826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:43.897609 826 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:43765
I20260928 16:42:43.897624 892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:43765 every 8 connection(s)
I20260928 16:42:43.899365 826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:43.901996 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 826
I20260928 16:42:43.902199 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:43.902719 11231 external_mini_cluster.cc:1428] Setting key 0dae92b26887a10ed2de9fbf54c2ca3b
I20260928 16:42:43.904517 893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:43.913059 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:43.904870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34025 (local address 127.10.247.254:43765)
0928 16:42:43.905824 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:43.905842 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:43.905901 (+ 59us) server_negotiation.cc:407] Connection header received
0928 16:42:43.908194 (+ 2293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:43.908231 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:43.908842 (+ 611us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:43.909405 (+ 563us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:43.910576 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:43.910585 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:43.910603 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:43.910621 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:43.910719 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:43.910761 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:43.911029 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:43.911180 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 16:42:43.911442 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":365,"threads_started":1}
I20260928 16:42:43.918056 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:43.920595 893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84: Bootstrap starting.
I20260928 16:42:43.925539 893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:43.926189 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:43.923165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42539 (local address 127.10.247.254:43765)
0928 16:42:43.923358 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:43.923363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:43.923930 (+ 567us) server_negotiation.cc:407] Connection header received
0928 16:42:43.924113 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:43.924118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:43.924204 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:43.924330 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:43.925417 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:43.925423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:43.925428 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:43.925452 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:43.925521 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:43.925545 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:43.925674 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:43.925757 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 16:42:43.925835 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:42:43.927228 893 log.cc:824] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:43.931524 893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84: No bootstrap required, opened a new log
I20260928 16:42:43.940881 893 raft_consensus.cc:348] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50c6686374c4590a356c2312a4a8d84" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43765 } }
I20260928 16:42:43.941371 893 raft_consensus.cc:374] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:43.941453 893 raft_consensus.cc:729] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50c6686374c4590a356c2312a4a8d84, State: Initialized, Role: FOLLOWER
I20260928 16:42:43.942085 893 consensus_queue.cc:260] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50c6686374c4590a356c2312a4a8d84" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43765 } }
I20260928 16:42:43.942322 893 raft_consensus.cc:388] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:43.942430 893 raft_consensus.cc:482] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:43.942570 893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:43.944919 893 raft_consensus.cc:504] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50c6686374c4590a356c2312a4a8d84" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43765 } }
I20260928 16:42:43.945452 893 leader_election.cc:302] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d50c6686374c4590a356c2312a4a8d84; no voters:
I20260928 16:42:43.946054 893 leader_election.cc:288] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:43.946235 903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:43.947208 903 raft_consensus.cc:686] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [term 1 LEADER]: Becoming Leader. State: Replica: d50c6686374c4590a356c2312a4a8d84, State: Running, Role: LEADER
I20260928 16:42:43.947804 903 consensus_queue.cc:237] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50c6686374c4590a356c2312a4a8d84" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43765 } }
I20260928 16:42:43.948431 893 sys_catalog.cc:564] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:43.951030 904 sys_catalog.cc:455] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d50c6686374c4590a356c2312a4a8d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50c6686374c4590a356c2312a4a8d84" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43765 } } }
I20260928 16:42:43.951108 905 sys_catalog.cc:455] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d50c6686374c4590a356c2312a4a8d84. Latest consensus state: current_term: 1 leader_uuid: "d50c6686374c4590a356c2312a4a8d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50c6686374c4590a356c2312a4a8d84" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 43765 } } }
I20260928 16:42:43.951529 904 sys_catalog.cc:458] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:43.951529 905 sys_catalog.cc:458] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:43.953639 911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:43.960537 911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:43.971426 911 catalog_manager.cc:1347] Generated new cluster ID: 2ca57d0cb035413fb7b95c564e8c00e8
I20260928 16:42:43.971560 911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:43.981971 911 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:43.982513 911 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:43.991612 911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d50c6686374c4590a356c2312a4a8d84: Generated new TSK 0
I20260928 16:42:43.993098 911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:44.009496 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 826
W20260928 16:42:44.011109 888 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34025
W20260928 16:42:44.012074 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 826: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:44.012245 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 826
2026-09-28T16:42:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1983 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:44 dist-test-slave-9fn6 krb5kdc[924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:44 dist-test-slave-9fn6 krb5kdc[924](info): set up 2 sockets
Sep 28 16:42:44 dist-test-slave-9fn6 krb5kdc[924](info): commencing operation
krb5kdc: starting...
W20260928 16:42:45.069392 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:45 dist-test-slave-9fn6 krb5kdc[924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:42:45.399418 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:40273
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:39301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:40273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:45.788681 937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:45.789096 937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:45.789247 937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:45.797091 937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:45.797235 937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:45.797317 937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:45.797406 937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:45.797464 937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:45.808737 937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:39301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:40273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:40273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:45.810545 937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:45.812214 937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:45.825881 942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:45.825855 945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:45.825778 943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:45.826846 937 server_base.cc:1034] running on GCE node
I20260928 16:42:45.827746 937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:45.829485 937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:45.830703 937 hybrid_clock.cc:648] HybridClock initialized: now 1790613765830655 us; error 39 us; skew 500 ppm
I20260928 16:42:45.831072 937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:45 dist-test-slave-9fn6 krb5kdc[924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613765, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:45.834565 937 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:42:45.836251 937 webserver.cc:460] Webserver started at http://127.10.247.254:43529/ using document root <none> and password file <none>
I20260928 16:42:45.837109 937 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:45.837239 937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:45.837563 937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:45.841177 937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "ec65f51a1dae4daebf85fcda0fd259fc"
format_stamp: "Formatted at 2026-09-28 16:42:45 on dist-test-slave-9fn6"
server_key: "4a58dfc61c41ddeb9c55924ee7140b14"
server_key_iv: "2769dc6ae35a35bf4fc047440d41d158"
server_key_version: "encryptionkey@0"
I20260928 16:42:45.842048 937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "ec65f51a1dae4daebf85fcda0fd259fc"
format_stamp: "Formatted at 2026-09-28 16:42:45 on dist-test-slave-9fn6"
server_key: "4a58dfc61c41ddeb9c55924ee7140b14"
server_key_iv: "2769dc6ae35a35bf4fc047440d41d158"
server_key_version: "encryptionkey@0"
I20260928 16:42:45.848376 937 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:42:45.852382 952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:45.853904 937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:42:45.854089 937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "ec65f51a1dae4daebf85fcda0fd259fc"
format_stamp: "Formatted at 2026-09-28 16:42:45 on dist-test-slave-9fn6"
server_key: "4a58dfc61c41ddeb9c55924ee7140b14"
server_key_iv: "2769dc6ae35a35bf4fc047440d41d158"
server_key_version: "encryptionkey@0"
I20260928 16:42:45.854300 937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:45.889205 937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:45.894296 937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:45.894619 937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:45.914599 937 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:40273
I20260928 16:42:45.914595 1003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:40273 every 8 connection(s)
I20260928 16:42:45.916224 937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:45.920847 1004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:45.922331 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 937
I20260928 16:42:45.922528 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:45.923041 11231 external_mini_cluster.cc:1428] Setting key 6072f5ec366bf7c1b67fb864cd3e213e
I20260928 16:42:45.937515 1004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc: Bootstrap starting.
Sep 28 16:42:45 dist-test-slave-9fn6 krb5kdc[924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:45.941781 1004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:45.943138 1004 log.cc:824] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:45.946671 1004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc: No bootstrap required, opened a new log
I20260928 16:42:45.949635 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:45.925103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52377 (local address 127.10.247.254:40273)
0928 16:42:45.925998 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:45.926018 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:45.926067 (+ 49us) server_negotiation.cc:407] Connection header received
0928 16:42:45.927853 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:45.927888 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:45.928465 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:45.928970 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:45.931043 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:45.933140 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:45.935137 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:45.936344 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:45.936556 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:45.941594 (+ 5038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:45.941604 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:45.941623 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:45.941675 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:45.945065 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:45.945834 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:45.945842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:45.945849 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:45.946038 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:45.946438 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:45.946443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:45.946445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:45.947059 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:45.947402 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:45.947831 (+ 429us) server_negotiation.cc:299] Negotiation successful
0928 16:42:45.948356 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":397,"threads_started":1}
I20260928 16:42:45.954090 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:45.956055 1004 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec65f51a1dae4daebf85fcda0fd259fc" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 40273 } }
I20260928 16:42:45.956466 1004 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:45.956561 1004 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec65f51a1dae4daebf85fcda0fd259fc, State: Initialized, Role: FOLLOWER
I20260928 16:42:45.957264 1004 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec65f51a1dae4daebf85fcda0fd259fc" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 40273 } }
I20260928 16:42:45.957576 1004 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:45.957720 1004 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:45.957927 1004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:45.960884 1004 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec65f51a1dae4daebf85fcda0fd259fc" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 40273 } }
I20260928 16:42:45.961493 1004 leader_election.cc:302] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec65f51a1dae4daebf85fcda0fd259fc; no voters:
I20260928 16:42:45.962246 1004 leader_election.cc:288] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:45.962343 1014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:45.963531 1014 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [term 1 LEADER]: Becoming Leader. State: Replica: ec65f51a1dae4daebf85fcda0fd259fc, State: Running, Role: LEADER
I20260928 16:42:45.964143 1014 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec65f51a1dae4daebf85fcda0fd259fc" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 40273 } }
I20260928 16:42:45.964941 1004 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:45.967661 1016 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec65f51a1dae4daebf85fcda0fd259fc. Latest consensus state: current_term: 1 leader_uuid: "ec65f51a1dae4daebf85fcda0fd259fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec65f51a1dae4daebf85fcda0fd259fc" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 40273 } } }
I20260928 16:42:45.968068 1016 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:45.968667 1015 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec65f51a1dae4daebf85fcda0fd259fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec65f51a1dae4daebf85fcda0fd259fc" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 40273 } } }
I20260928 16:42:45.968974 1015 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:45.971428 1021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:45.977702 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:45.958937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58017 (local address 127.10.247.254:40273)
0928 16:42:45.959150 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:45.959156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:45.959624 (+ 468us) server_negotiation.cc:407] Connection header received
0928 16:42:45.959774 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:45.959781 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:45.959874 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:45.959986 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:45.961683 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:45.962897 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:45.964721 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:45.965406 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:45.965548 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:45.969596 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:45.969604 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:45.969613 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:45.969671 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:45.974096 (+ 4425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:45.975229 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:45.975235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:45.975239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:45.975411 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:45.975900 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:45.975904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:45.975907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:45.976313 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:45.976586 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:45.977030 (+ 444us) server_negotiation.cc:299] Negotiation successful
0928 16:42:45.977300 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 16:42:45.980237 1021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:45.991549 1021 catalog_manager.cc:1347] Generated new cluster ID: 2132df77ccad42a6ae7809c6377d385a
I20260928 16:42:45.991693 1021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:46.016080 1021 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:46.017809 1021 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:46.029186 1021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec65f51a1dae4daebf85fcda0fd259fc: Generated new TSK 0
I20260928 16:42:46.030229 1021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:46.041551 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 937
I20260928 16:42:46.043191 999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:46.146971 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 937
2026-09-28T16:42:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2136 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:46 dist-test-slave-9fn6 krb5kdc[1036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:46 dist-test-slave-9fn6 krb5kdc[1036](info): set up 2 sockets
Sep 28 16:42:46 dist-test-slave-9fn6 krb5kdc[1036](info): commencing operation
krb5kdc: starting...
W20260928 16:42:47.201407 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:47 dist-test-slave-9fn6 krb5kdc[1036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:42:47.524669 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:41573
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:44875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:41573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:47.914678 1049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:47.915127 1049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:47.915290 1049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:47.923009 1049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:47.923167 1049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:47.923254 1049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:47.923345 1049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:47.923411 1049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:47.936672 1049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:44875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:41573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:41573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:47.938460 1049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:47.940171 1049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:47.954164 1057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:47.954228 1054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:47.954205 1055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:47.955116 1049 server_base.cc:1034] running on GCE node
I20260928 16:42:47.956108 1049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:47.957964 1049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:47.959244 1049 hybrid_clock.cc:648] HybridClock initialized: now 1790613767959155 us; error 71 us; skew 500 ppm
I20260928 16:42:47.959677 1049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:47 dist-test-slave-9fn6 krb5kdc[1036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613767, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:47.967051 1049 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:42:47.968971 1049 webserver.cc:460] Webserver started at http://127.10.247.254:36417/ using document root <none> and password file <none>
I20260928 16:42:47.969859 1049 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:47.970000 1049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:47.970355 1049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:47.974157 1049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "86f20a5ffd434f1685ca221f9fdf8acb"
format_stamp: "Formatted at 2026-09-28 16:42:47 on dist-test-slave-9fn6"
server_key: "173e607254c7826def99c600656b7670"
server_key_iv: "969d22ceb541a0721bf25b9da48c30a1"
server_key_version: "encryptionkey@0"
I20260928 16:42:47.975116 1049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "86f20a5ffd434f1685ca221f9fdf8acb"
format_stamp: "Formatted at 2026-09-28 16:42:47 on dist-test-slave-9fn6"
server_key: "173e607254c7826def99c600656b7670"
server_key_iv: "969d22ceb541a0721bf25b9da48c30a1"
server_key_version: "encryptionkey@0"
I20260928 16:42:47.981590 1049 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260928 16:42:47.985873 1064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:47.987457 1049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 16:42:47.987663 1049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "86f20a5ffd434f1685ca221f9fdf8acb"
format_stamp: "Formatted at 2026-09-28 16:42:47 on dist-test-slave-9fn6"
server_key: "173e607254c7826def99c600656b7670"
server_key_iv: "969d22ceb541a0721bf25b9da48c30a1"
server_key_version: "encryptionkey@0"
I20260928 16:42:47.987885 1049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:48.006553 1049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:48.011853 1049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:48.012183 1049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:48.032474 1049 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:41573
I20260928 16:42:48.032481 1115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:41573 every 8 connection(s)
I20260928 16:42:48.034122 1049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:48.038625 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 1049
I20260928 16:42:48.038686 1116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:48.038810 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:48.039358 11231 external_mini_cluster.cc:1428] Setting key 3d144a587eeda847c5b3ec2a4f415c5a
I20260928 16:42:48.053542 1116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb: Bootstrap starting.
I20260928 16:42:48.053561 1119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:48.041325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59383 (local address 127.10.247.254:41573)
0928 16:42:48.042213 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:48.042227 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:48.042276 (+ 49us) server_negotiation.cc:407] Connection header received
0928 16:42:48.044047 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:48.044075 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:48.044514 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:48.044958 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:48.045575 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:48.046992 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:48.048813 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:48.049520 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:48.049673 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:48.050995 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:48.051003 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:48.051018 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:48.051681 (+ 663us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:48.051771 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:48.051799 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:48.051887 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:48.052119 (+ 232us) server_negotiation.cc:299] Negotiation successful
0928 16:42:48.052390 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":373,"threads_started":1}
I20260928 16:42:48.057524 1116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:48.057870 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:48.058997 1116 log.cc:824] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:48.062953 1116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb: No bootstrap required, opened a new log
I20260928 16:42:48.070837 1119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:48.063518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53565 (local address 127.10.247.254:41573)
0928 16:42:48.063725 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:48.063732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:48.064233 (+ 501us) server_negotiation.cc:407] Connection header received
0928 16:42:48.064420 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:48.064426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:48.064518 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:48.064671 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:48.065302 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:48.066467 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:48.067911 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:48.068693 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:48.068834 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:48.069942 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:48.069949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:48.069954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:48.069979 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:48.070043 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:48.070066 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:48.070185 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:48.070347 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 16:42:48.070451 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:42:48.073153 1116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f20a5ffd434f1685ca221f9fdf8acb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41573 } }
I20260928 16:42:48.073602 1116 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:48.073693 1116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86f20a5ffd434f1685ca221f9fdf8acb, State: Initialized, Role: FOLLOWER
I20260928 16:42:48.074376 1116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f20a5ffd434f1685ca221f9fdf8acb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41573 } }
I20260928 16:42:48.074640 1116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:48.074779 1116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:48.074946 1116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:48.077787 1116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f20a5ffd434f1685ca221f9fdf8acb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41573 } }
I20260928 16:42:48.078397 1116 leader_election.cc:302] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86f20a5ffd434f1685ca221f9fdf8acb; no voters:
I20260928 16:42:48.079026 1116 leader_election.cc:288] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:48.079209 1127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:48.079896 1127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [term 1 LEADER]: Becoming Leader. State: Replica: 86f20a5ffd434f1685ca221f9fdf8acb, State: Running, Role: LEADER
I20260928 16:42:48.080521 1127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f20a5ffd434f1685ca221f9fdf8acb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41573 } }
I20260928 16:42:48.081475 1116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:48.083478 1129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86f20a5ffd434f1685ca221f9fdf8acb. Latest consensus state: current_term: 1 leader_uuid: "86f20a5ffd434f1685ca221f9fdf8acb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f20a5ffd434f1685ca221f9fdf8acb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41573 } } }
I20260928 16:42:48.083992 1129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:48.084969 1135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:48.084758 1128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86f20a5ffd434f1685ca221f9fdf8acb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f20a5ffd434f1685ca221f9fdf8acb" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41573 } } }
I20260928 16:42:48.085124 1128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:48.090759 1135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:48.101890 1135 catalog_manager.cc:1347] Generated new cluster ID: 78073c9a8d9640e0993b9a47b7996c94
I20260928 16:42:48.102033 1135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:48.117700 1135 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:48.119349 1135 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:48.126566 1135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86f20a5ffd434f1685ca221f9fdf8acb: Generated new TSK 0
I20260928 16:42:48.127964 1135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:48.140584 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1049
W20260928 16:42:48.141863 1111 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59383
W20260928 16:42:48.142717 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1049: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:48.142897 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 1049
2026-09-28T16:42:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1995 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:48 dist-test-slave-9fn6 krb5kdc[1148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:48 dist-test-slave-9fn6 krb5kdc[1148](info): set up 2 sockets
Sep 28 16:42:48 dist-test-slave-9fn6 krb5kdc[1148](info): commencing operation
krb5kdc: starting...
W20260928 16:42:49.205416 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:49 dist-test-slave-9fn6 krb5kdc[1148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:42:49.527935 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37651
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:49.926031 1161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:49.926606 1161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:49.926810 1161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:49.937733 1161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:49.938012 1161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:49.938151 1161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:49.938258 1161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:49.938324 1161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:49.950750 1161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:49.952473 1161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:49.954149 1161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:49.967757 1166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:49.967710 1167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:49.967782 1169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:49.974148 1161 server_base.cc:1034] running on GCE node
I20260928 16:42:49.975328 1161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:49.977550 1161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:49.978839 1161 hybrid_clock.cc:648] HybridClock initialized: now 1790613769978772 us; error 55 us; skew 500 ppm
I20260928 16:42:49.979324 1161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:49 dist-test-slave-9fn6 krb5kdc[1148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613769, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:49.983142 1161 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:42:49.985194 1161 webserver.cc:460] Webserver started at http://127.10.247.254:37719/ using document root <none> and password file <none>
I20260928 16:42:49.986140 1161 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:49.986284 1161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:49.986644 1161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:49.990615 1161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "71b39edce53e4a7199e4e31aec4d821b"
format_stamp: "Formatted at 2026-09-28 16:42:49 on dist-test-slave-9fn6"
server_key: "dfb87eaf73e85c29697bff1c439b17f5"
server_key_iv: "67aca1521596c4251248f8238e234c2f"
server_key_version: "encryptionkey@0"
I20260928 16:42:49.991611 1161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "71b39edce53e4a7199e4e31aec4d821b"
format_stamp: "Formatted at 2026-09-28 16:42:49 on dist-test-slave-9fn6"
server_key: "dfb87eaf73e85c29697bff1c439b17f5"
server_key_iv: "67aca1521596c4251248f8238e234c2f"
server_key_version: "encryptionkey@0"
I20260928 16:42:49.998454 1161 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260928 16:42:50.003144 1176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:50.005272 1161 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260928 16:42:50.005499 1161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "71b39edce53e4a7199e4e31aec4d821b"
format_stamp: "Formatted at 2026-09-28 16:42:49 on dist-test-slave-9fn6"
server_key: "dfb87eaf73e85c29697bff1c439b17f5"
server_key_iv: "67aca1521596c4251248f8238e234c2f"
server_key_version: "encryptionkey@0"
I20260928 16:42:50.005772 1161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:50.013142 1161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:50.018080 1161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:50.018404 1161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:50.038708 1161 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:37651
I20260928 16:42:50.038735 1227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:37651 every 8 connection(s)
I20260928 16:42:50.040405 1161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:50.044915 1228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:50.046383 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 1161
I20260928 16:42:50.046603 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:50.047226 11231 external_mini_cluster.cc:1428] Setting key f592548559c276034351d53669b13ddf
I20260928 16:42:50.056093 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:50.049390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43501 (local address 127.10.247.254:37651)
0928 16:42:50.050324 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:50.050340 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:50.050392 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:42:50.052186 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:50.052217 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:50.052684 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:50.053080 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:50.054296 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:50.054303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:50.054318 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:50.054333 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:50.054399 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:50.054424 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:50.054625 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:50.054731 (+ 106us) server_negotiation.cc:299] Negotiation successful
0928 16:42:50.054973 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":440,"threads_started":1}
I20260928 16:42:50.060070 1228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b: Bootstrap starting.
I20260928 16:42:50.060393 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:50.063934 1228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:50.065335 1228 log.cc:824] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:50.068251 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:50.065416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35303 (local address 127.10.247.254:37651)
0928 16:42:50.065601 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:50.065609 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:50.066158 (+ 549us) server_negotiation.cc:407] Connection header received
0928 16:42:50.066358 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:50.066362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:50.066425 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:50.066511 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:50.067579 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:50.067584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:50.067589 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:42:50.067608 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:50.067660 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:42:50.067682 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:50.067797 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:50.067870 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 16:42:50.067938 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:42:50.069331 1228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b: No bootstrap required, opened a new log
I20260928 16:42:50.077706 1228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b39edce53e4a7199e4e31aec4d821b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37651 } }
I20260928 16:42:50.078195 1228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:42:50.078280 1228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b39edce53e4a7199e4e31aec4d821b, State: Initialized, Role: FOLLOWER
I20260928 16:42:50.078899 1228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b39edce53e4a7199e4e31aec4d821b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37651 } }
I20260928 16:42:50.079128 1228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:42:50.079236 1228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:42:50.079372 1228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:50.082026 1228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b39edce53e4a7199e4e31aec4d821b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37651 } }
I20260928 16:42:50.082664 1228 leader_election.cc:302] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71b39edce53e4a7199e4e31aec4d821b; no voters:
I20260928 16:42:50.083290 1228 leader_election.cc:288] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:42:50.083446 1238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:50.084197 1238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [term 1 LEADER]: Becoming Leader. State: Replica: 71b39edce53e4a7199e4e31aec4d821b, State: Running, Role: LEADER
I20260928 16:42:50.084858 1238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b39edce53e4a7199e4e31aec4d821b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37651 } }
I20260928 16:42:50.085842 1228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:50.088258 1240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71b39edce53e4a7199e4e31aec4d821b. Latest consensus state: current_term: 1 leader_uuid: "71b39edce53e4a7199e4e31aec4d821b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b39edce53e4a7199e4e31aec4d821b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37651 } } }
I20260928 16:42:50.088717 1240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:50.090308 1245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:50.090164 1239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71b39edce53e4a7199e4e31aec4d821b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b39edce53e4a7199e4e31aec4d821b" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 37651 } } }
I20260928 16:42:50.090471 1239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:50.096432 1245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:42:50.108160 1245 catalog_manager.cc:1347] Generated new cluster ID: 7c9d11d16a7f479c9d4a1447363dca90
I20260928 16:42:50.108307 1245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:50.131613 1245 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:50.132280 1245 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:50.159454 1245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71b39edce53e4a7199e4e31aec4d821b: Generated new TSK 0
I20260928 16:42:50.160676 1245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:42:50.169739 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1161
W20260928 16:42:50.171037 1223 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43501
W20260928 16:42:50.171828 11231 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1161: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:42:50.171983 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 1161
2026-09-28T16:42:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2029 ms)
[----------] 18 tests from AuthTokenIssuingTest (37161 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:50 dist-test-slave-9fn6 krb5kdc[1259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:50 dist-test-slave-9fn6 krb5kdc[1259](info): set up 2 sockets
Sep 28 16:42:50 dist-test-slave-9fn6 krb5kdc[1259](info): commencing operation
krb5kdc: starting...
W20260928 16:42:51.233628 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:51 dist-test-slave-9fn6 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:42:51.561388 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:41425
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:41425,127.10.247.253:33955,127.10.247.252:37565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:51.959825 1272 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:42:51.960269 1272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:51.960383 1272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:51.960502 1272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:51.968509 1272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:51.968675 1272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:51.968775 1272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:51.968851 1272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:51.968943 1272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:51.980201 1272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:41425,127.10.247.253:33955,127.10.247.252:37565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:41425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:51.981999 1272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:51.983714 1272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:51.997502 1277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:51.997587 1280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:51.997537 1278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:51.999044 1272 server_base.cc:1034] running on GCE node
I20260928 16:42:52.000048 1272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:52.002010 1272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:52.003266 1272 hybrid_clock.cc:648] HybridClock initialized: now 1790613772003194 us; error 52 us; skew 500 ppm
I20260928 16:42:52.003675 1272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:52 dist-test-slave-9fn6 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613772, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:52.007416 1272 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:42:52.009800 1272 webserver.cc:460] Webserver started at http://127.10.247.254:36049/ using document root <none> and password file <none>
I20260928 16:42:52.010777 1272 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:52.010910 1272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:52.011332 1272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:52.015347 1272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "5f71d0ef906644f78afd9fa2069bad77"
format_stamp: "Formatted at 2026-09-28 16:42:52 on dist-test-slave-9fn6"
server_key: "01a04d1846aa718d27d85350c52161d3"
server_key_iv: "79acdf26092ecc4d4226edf673301fe8"
server_key_version: "encryptionkey@0"
I20260928 16:42:52.016398 1272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "5f71d0ef906644f78afd9fa2069bad77"
format_stamp: "Formatted at 2026-09-28 16:42:52 on dist-test-slave-9fn6"
server_key: "01a04d1846aa718d27d85350c52161d3"
server_key_iv: "79acdf26092ecc4d4226edf673301fe8"
server_key_version: "encryptionkey@0"
I20260928 16:42:52.023115 1272 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:42:52.027555 1287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:52.029364 1272 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:42:52.029564 1272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "5f71d0ef906644f78afd9fa2069bad77"
format_stamp: "Formatted at 2026-09-28 16:42:52 on dist-test-slave-9fn6"
server_key: "01a04d1846aa718d27d85350c52161d3"
server_key_iv: "79acdf26092ecc4d4226edf673301fe8"
server_key_version: "encryptionkey@0"
I20260928 16:42:52.029819 1272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:52.048345 1272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:52.053447 1272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:52.053761 1272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:52.075188 1272 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:41425
I20260928 16:42:52.075214 1338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:41425 every 8 connection(s)
I20260928 16:42:52.076972 1272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:52.081775 1339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:52.086150 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 1272
I20260928 16:42:52.086414 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:52.087294 11231 external_mini_cluster.cc:1428] Setting key 2b8a67326c805ba70df2797aef0b4bf9
I20260928 16:42:52.091785 1339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:42:52 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613772, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:52.119689 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.096588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54130 (local address 127.10.247.254:41425)
0928 16:42:52.098726 (+ 2138us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:52.098734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:52.098803 (+ 69us) server_negotiation.cc:407] Connection header received
0928 16:42:52.100582 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:52.100597 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:52.100952 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:52.101174 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:52.102727 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.104070 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.106040 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.106703 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.106860 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:52.111266 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:52.111277 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:52.111294 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:52.111334 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:52.114500 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:52.115308 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:52.115317 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:52.115320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:52.115495 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:52.115897 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:52.115901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:52.115903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:52.116576 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:52.116847 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:52.117440 (+ 593us) server_negotiation.cc:299] Negotiation successful
0928 16:42:52.117901 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":1851,"spinlock_wait_cycles":22272,"thread_start_us":1568,"threads_started":1}
I20260928 16:42:52.119711 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.096595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:41425 (local address 127.0.0.1:54130)
0928 16:42:52.097665 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.097733 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:52.100133 (+ 2400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:52.101217 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:52.101226 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:52.102078 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:52.102503 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:52.102523 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.104225 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.104231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:52.105883 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:52.105898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.106972 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.106978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:52.107156 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:52.107946 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:52.107978 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:52.110985 (+ 3007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:52.114680 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:52.114694 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:52.114708 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:52.115158 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:52.115605 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:52.115611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:52.115614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:52.115786 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:52.116743 (+ 957us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:52.116756 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:52.117019 (+ 263us) client_negotiation.cc:769] Sending connection context
0928 16:42:52.117281 (+ 262us) client_negotiation.cc:240] Negotiation successful
0928 16:42:52.118043 (+ 762us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":160,"thread_start_us":464,"threads_started":1}
WARNING: no policy specified for kudu/127.10.247.253@KRBTEST.COM; defaulting to no policy
I20260928 16:42:52.123786 1339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:42:52.125227 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.124738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47910)
0928 16:42:52.124926 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.125054 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260928 16:42:52.125712 1289 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.253@KRBTEST.COM" created.
I20260928 16:42:52.127622 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.127143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47912)
0928 16:42:52.127326 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.127424 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260928 16:42:52.127867 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:33955: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab.
Entry for principal kudu/127.10.247.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab.
I20260928 16:42:52.162274 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.253:33955
--webserver_interface=127.10.247.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab
--principal=kudu/127.10.247.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:41425,127.10.247.253:33955,127.10.247.252:37565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:42:52.177316 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } attempt: 1
I20260928 16:42:52.178929 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.178324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47920)
0928 16:42:52.178576 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.178748 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260928 16:42:52.180222 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.179684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47936)
0928 16:42:52.179937 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.180050 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260928 16:42:52.180464 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:33955: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
I20260928 16:42:52.248754 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } attempt: 2
I20260928 16:42:52.250222 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.249602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47944)
0928 16:42:52.249915 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.250023 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260928 16:42:52.251295 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.250903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47960)
0928 16:42:52.251056 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.251151 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260928 16:42:52.251483 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:33955: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
I20260928 16:42:52.342758 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } attempt: 3
I20260928 16:42:52.344206 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.343630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47968)
0928 16:42:52.343876 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.344029 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260928 16:42:52.345412 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.344986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47980)
0928 16:42:52.345155 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.345235 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260928 16:42:52.345603 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:33955: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
I20260928 16:42:52.488879 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } attempt: 4
I20260928 16:42:52.490371 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.489718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47982)
0928 16:42:52.490086 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.490201 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260928 16:42:52.491515 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.491102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:47994)
0928 16:42:52.491268 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.491366 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260928 16:42:52.491742 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:33955: Network error: Client connection negotiation failed: client connection to 127.10.247.253:33955: connect: Connection refused (error 111)
W20260928 16:42:52.551810 1346 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:42:52.552263 1346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:52.552367 1346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:52.552485 1346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:52.560730 1346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:52.560870 1346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:52.560966 1346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:52.561045 1346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:52.561137 1346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:52.572330 1346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:41425,127.10.247.253:33955,127.10.247.252:37565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab
--principal=kudu/127.10.247.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.253:33955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.247.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:52.574112 1346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:52.575809 1346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:52.589603 1351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:52.589576 1354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:52.589576 1352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:52.590368 1346 server_base.cc:1034] running on GCE node
I20260928 16:42:52.591320 1346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:52.592993 1346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:52.594230 1346 hybrid_clock.cc:648] HybridClock initialized: now 1790613772594157 us; error 48 us; skew 500 ppm
I20260928 16:42:52.594585 1346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:52 dist-test-slave-9fn6 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613772, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:52.598184 1346 init.cc:377] Logged in from keytab as kudu/127.10.247.253@KRBTEST.COM (short username kudu)
I20260928 16:42:52.599912 1346 webserver.cc:460] Webserver started at http://127.10.247.253:38429/ using document root <none> and password file <none>
I20260928 16:42:52.600790 1346 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:52.600900 1346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:52.601260 1346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:52.604813 1346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/data/instance:
uuid: "43e3ea1379854d6e9601f226c92c7152"
format_stamp: "Formatted at 2026-09-28 16:42:52 on dist-test-slave-9fn6"
server_key: "455853e9ec489f393271e91d338052cc"
server_key_iv: "4e66fef9deac0403888f73bc4a830ae4"
server_key_version: "encryptionkey@0"
I20260928 16:42:52.605736 1346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/wal/instance:
uuid: "43e3ea1379854d6e9601f226c92c7152"
format_stamp: "Formatted at 2026-09-28 16:42:52 on dist-test-slave-9fn6"
server_key: "455853e9ec489f393271e91d338052cc"
server_key_iv: "4e66fef9deac0403888f73bc4a830ae4"
server_key_version: "encryptionkey@0"
I20260928 16:42:52.611732 1346 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260928 16:42:52.615522 1361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:52.617204 1346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:42:52.617375 1346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/wal
uuid: "43e3ea1379854d6e9601f226c92c7152"
format_stamp: "Formatted at 2026-09-28 16:42:52 on dist-test-slave-9fn6"
server_key: "455853e9ec489f393271e91d338052cc"
server_key_iv: "4e66fef9deac0403888f73bc4a830ae4"
server_key_version: "encryptionkey@0"
I20260928 16:42:52.617601 1346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:52.638417 1346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:52.643607 1346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:52.643920 1346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:52.664625 1346 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.253:33955
I20260928 16:42:52.664676 1412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.253:33955 every 8 connection(s)
I20260928 16:42:52.666313 1346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
I20260928 16:42:52.667466 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 1346
I20260928 16:42:52.667680 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-1/wal/instance
I20260928 16:42:52.668267 11231 external_mini_cluster.cc:1428] Setting key 6f7279c3c662b513185bc33719aa78e6
I20260928 16:42:52.671973 1413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:52.682313 1413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:42:52 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613772, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.247.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.252@KRBTEST.COM" created.
I20260928 16:42:52.706975 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.686282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54146 (local address 127.10.247.254:41425)
0928 16:42:52.686893 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:52.686899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:52.688259 (+ 1360us) server_negotiation.cc:407] Connection header received
0928 16:42:52.690597 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:52.690604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:52.690706 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:52.690821 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:52.692911 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.693769 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.695527 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.695997 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.696138 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:52.700377 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:52.700383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:52.700387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:52.700434 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:52.703433 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:52.704368 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:52.704374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:52.704377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:52.704543 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:52.704985 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:52.704991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:52.704994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:52.705409 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:52.705554 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:52.706420 (+ 866us) server_negotiation.cc:299] Negotiation successful
0928 16:42:52.706660 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":319,"threads_started":1}
I20260928 16:42:52.708892 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.687067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:41425 (local address 127.0.0.1:54146)
0928 16:42:52.688033 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.688094 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:52.690241 (+ 2147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:52.691146 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:52.691157 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:52.691832 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:52.692605 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:52.692667 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.693956 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.693960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:52.695356 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:52.695371 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.696274 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.696283 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:52.696578 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:52.697425 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:52.697452 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:52.700090 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:52.703596 (+ 3506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:52.703608 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:52.703630 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:52.704194 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:52.704672 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:52.704676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:52.704679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:52.704883 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:52.705529 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:52.705537 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:52.705997 (+ 460us) client_negotiation.cc:769] Sending connection context
0928 16:42:52.706317 (+ 320us) client_negotiation.cc:240] Negotiation successful
0928 16:42:52.706983 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":402,"threads_started":1}
I20260928 16:42:52.711225 1413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:42:52 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613772, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
I20260928 16:42:52.728921 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.712286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:48010)
0928 16:42:52.712488 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.712507 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:52.712659 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:52.713523 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:52.713528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:52.714023 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:52.714232 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:52.714243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.715716 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.715721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:52.716934 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:52.716949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.717829 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.717835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:52.718017 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:52.718751 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:52.718774 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:52.721714 (+ 2940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:52.726038 (+ 4324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:52.726043 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:52.726046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:52.726470 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:52.726911 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:52.726915 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:52.726918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:52.727092 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:52.727816 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:52.727821 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:52.727972 (+ 151us) client_negotiation.cc:769] Sending connection context
0928 16:42:52.728103 (+ 131us) client_negotiation.cc:240] Negotiation successful
0928 16:42:52.728413 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260928 16:42:52.728951 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.712286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.10.247.253:33955)
0928 16:42:52.713038 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:52.713045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:52.713073 (+ 28us) server_negotiation.cc:407] Connection header received
0928 16:42:52.713122 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:52.713129 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:52.713357 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:52.713505 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:52.714404 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.715580 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.717064 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.717678 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.717778 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:52.721981 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:52.722001 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:52.722024 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:52.722067 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:52.725852 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:52.726638 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:52.726649 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:52.726653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:52.726804 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:52.727228 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:52.727233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:52.727235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:52.727683 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:52.727884 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:52.728251 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 16:42:52.728542 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":559,"thread_start_us":400,"threads_started":1}
I20260928 16:42:52.731979 1413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:42:52.733426 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.732875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47316)
0928 16:42:52.733063 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.733220 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260928 16:42:52.733753 1362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
I20260928 16:42:52.735594 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.735161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47328)
0928 16:42:52.735366 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.735458 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260928 16:42:52.735793 1413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab.
Entry for principal kudu/127.10.247.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab.
I20260928 16:42:52.750244 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.252:37565
--webserver_interface=127.10.247.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab
--principal=kudu/127.10.247.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:41425,127.10.247.253:33955,127.10.247.252:37565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:42:52.785161 1413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 1
I20260928 16:42:52.786703 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.786069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47334)
0928 16:42:52.786315 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.786465 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260928 16:42:52.787799 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.787399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47342)
0928 16:42:52.787557 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.787664 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 16:42:52.788004 1413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
I20260928 16:42:52.790988 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } attempt: 5
Sep 28 16:42:52 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613772, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
I20260928 16:42:52.804821 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.791734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48018 (local address 127.10.247.253:33955)
0928 16:42:52.791888 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:52.791895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:52.792013 (+ 118us) server_negotiation.cc:407] Connection header received
0928 16:42:52.792250 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:52.792254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:52.792325 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:52.792409 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:52.793347 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.794172 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.795368 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.795845 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.795954 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:52.799447 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:52.799452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:52.799455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:52.799493 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:52.802216 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:52.802889 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:52.802894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:52.802897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:52.803028 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:52.803386 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:52.803391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:52.803393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:52.803709 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:52.803856 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:52.804330 (+ 474us) server_negotiation.cc:299] Negotiation successful
0928 16:42:52.804524 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 16:42:52.805032 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.791741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:48018)
0928 16:42:52.791927 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.791946 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:52.792097 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:52.792466 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:52.792470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:52.792992 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:52.793204 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:52.793215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.794299 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.794305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:52.795225 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:52.795238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:52.795984 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:52.795989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:52.796193 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:52.796829 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:52.796850 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:52.799252 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:52.802352 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:52.802359 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:52.802364 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:52.802776 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:52.803131 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:52.803136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:52.803141 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:52.803286 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:52.803854 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:52.803860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:52.804066 (+ 206us) client_negotiation.cc:769] Sending connection context
0928 16:42:52.804242 (+ 176us) client_negotiation.cc:240] Negotiation successful
0928 16:42:52.804671 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 16:42:52.806272 1339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:42:52.807459 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.807057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47344)
0928 16:42:52.807225 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.807309 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260928 16:42:52.808350 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.808022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47346)
0928 16:42:52.808163 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.808250 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260928 16:42:52.808542 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
I20260928 16:42:52.856281 1413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 2
I20260928 16:42:52.857933 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.857271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47350)
0928 16:42:52.857540 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.857690 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260928 16:42:52.859144 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.858707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47358)
0928 16:42:52.858875 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.858976 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260928 16:42:52.859478 1413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
I20260928 16:42:52.859783 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 1
I20260928 16:42:52.860954 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.860418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47366)
0928 16:42:52.860660 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.860769 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260928 16:42:52.862174 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.861718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47372)
0928 16:42:52.861939 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.862045 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260928 16:42:52.862427 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
I20260928 16:42:52.930675 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 2
I20260928 16:42:52.932389 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.931734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47374)
0928 16:42:52.932048 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.932174 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260928 16:42:52.933723 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.933302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47380)
0928 16:42:52.933463 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.933554 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260928 16:42:52.933949 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
I20260928 16:42:52.950798 1413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 3
I20260928 16:42:52.952363 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.951719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47382)
0928 16:42:52.951975 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.952120 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260928 16:42:52.953651 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:52.953096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47384)
0928 16:42:52.953356 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:52.953462 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260928 16:42:52.953950 1413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
I20260928 16:42:53.040215 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 3
I20260928 16:42:53.041731 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.041162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47392)
0928 16:42:53.041405 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.041520 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260928 16:42:53.042901 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.042456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47408)
0928 16:42:53.042686 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.042768 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260928 16:42:53.043128 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
I20260928 16:42:53.097193 1413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 4
I20260928 16:42:53.098793 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.098080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47414)
0928 16:42:53.098387 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.098525 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260928 16:42:53.100085 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.099656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47422)
0928 16:42:53.099825 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.099906 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260928 16:42:53.100404 1413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
W20260928 16:42:53.151275 1422 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:42:53.151728 1422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:53.151834 1422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:53.151957 1422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:53.160085 1422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:53.160240 1422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:53.160334 1422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:53.160411 1422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:53.160509 1422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:53.171980 1422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:41425,127.10.247.253:33955,127.10.247.252:37565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab
--principal=kudu/127.10.247.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.252:37565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.247.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:53.173815 1422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:53.175571 1422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:53.189240 1430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:53.189281 1427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:53.189373 1428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:53.190680 1422 server_base.cc:1034] running on GCE node
I20260928 16:42:53.191692 1422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:53.193516 1422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:53.194758 1422 hybrid_clock.cc:648] HybridClock initialized: now 1790613773194698 us; error 40 us; skew 500 ppm
I20260928 16:42:53.195137 1422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:53 dist-test-slave-9fn6 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613773, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:53.202883 1422 init.cc:377] Logged in from keytab as kudu/127.10.247.252@KRBTEST.COM (short username kudu)
I20260928 16:42:53.205060 1422 webserver.cc:460] Webserver started at http://127.10.247.252:35265/ using document root <none> and password file <none>
I20260928 16:42:53.206002 1422 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:53.206132 1422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:53.206521 1422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:53.210383 1422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/data/instance:
uuid: "6dff920963e647b3a3e443a79ca08f5c"
format_stamp: "Formatted at 2026-09-28 16:42:53 on dist-test-slave-9fn6"
server_key: "4cf57eed2a510902aa5ff52f5fb7efac"
server_key_iv: "ae6cd3d10958e5be14a8c4430605a2d3"
server_key_version: "encryptionkey@0"
I20260928 16:42:53.211346 1422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/wal/instance:
uuid: "6dff920963e647b3a3e443a79ca08f5c"
format_stamp: "Formatted at 2026-09-28 16:42:53 on dist-test-slave-9fn6"
server_key: "4cf57eed2a510902aa5ff52f5fb7efac"
server_key_iv: "ae6cd3d10958e5be14a8c4430605a2d3"
server_key_version: "encryptionkey@0"
I20260928 16:42:53.217650 1422 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260928 16:42:53.220383 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 4
I20260928 16:42:53.221798 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.221223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47428)
0928 16:42:53.221437 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.221568 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260928 16:42:53.222082 1437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:53.223141 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.222700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47434)
0928 16:42:53.222858 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.222956 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 16:42:53.223341 1339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:37565: Network error: Client connection negotiation failed: client connection to 127.10.247.252:37565: connect: Connection refused (error 111)
I20260928 16:42:53.224047 1422 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260928 16:42:53.224232 1422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/wal
uuid: "6dff920963e647b3a3e443a79ca08f5c"
format_stamp: "Formatted at 2026-09-28 16:42:53 on dist-test-slave-9fn6"
server_key: "4cf57eed2a510902aa5ff52f5fb7efac"
server_key_iv: "ae6cd3d10958e5be14a8c4430605a2d3"
server_key_version: "encryptionkey@0"
I20260928 16:42:53.224469 1422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:53.249058 1422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:53.254164 1422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:53.254477 1422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:53.274374 1422 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.252:37565
I20260928 16:42:53.274395 1489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.252:37565 every 8 connection(s)
I20260928 16:42:53.276047 1422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
I20260928 16:42:53.280797 1490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:53.284229 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 1422
I20260928 16:42:53.284458 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790613388455587-11231-0/minicluster-data/master-2/wal/instance
I20260928 16:42:53.285117 11231 external_mini_cluster.cc:1428] Setting key 66df54c7007b23288075df05759dc586
I20260928 16:42:53.290395 1490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:42:53 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
Sep 28 16:42:53 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613773, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:53.310174 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.293287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54154 (local address 127.10.247.254:41425)
0928 16:42:53.293856 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.293861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.294555 (+ 694us) server_negotiation.cc:407] Connection header received
0928 16:42:53.294766 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.294771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.294847 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.294951 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.296553 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.297569 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.299676 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.300118 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.300235 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.303859 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.303864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.303868 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.303910 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.306704 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.307413 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.307419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.307421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.307559 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.307949 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.307955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.307957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.308265 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.308407 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.309213 (+ 806us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.309473 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":287,"threads_started":1}
I20260928 16:42:53.311112 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.293307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:41425 (local address 127.0.0.1:54154)
0928 16:42:53.294401 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.294442 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:53.294615 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:53.295047 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:53.295054 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:53.295969 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:53.296340 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.296353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.297759 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.297763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.299494 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.299514 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.300374 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.300382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.300529 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.301164 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:53.301230 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:53.303621 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:53.306840 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.306854 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.306871 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.307293 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.307666 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.307671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.307675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.307842 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.308389 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:53.308398 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:53.308792 (+ 394us) client_negotiation.cc:769] Sending connection context
0928 16:42:53.309073 (+ 281us) client_negotiation.cc:240] Negotiation successful
0928 16:42:53.309702 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":3,"thread_start_us":684,"threads_started":1}
I20260928 16:42:53.311131 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.287556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47448 (local address 127.10.247.252:37565)
0928 16:42:53.288475 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.288490 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.288541 (+ 51us) server_negotiation.cc:407] Connection header received
0928 16:42:53.290442 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.290473 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.291007 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.291467 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.293269 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.295248 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.297165 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.298618 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.298790 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.303669 (+ 4879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.303679 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.303697 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.303737 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.306766 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.307580 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.307588 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.307590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.307759 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.308147 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.308151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.308153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.308852 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.309191 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.309483 (+ 292us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.309834 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":658,"spinlock_wait_cycles":36224,"thread_start_us":394,"threads_started":1}
I20260928 16:42:53.312975 1490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:42:53 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613773, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
Sep 28 16:42:53 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:53.328711 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.313901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:33955 (local address 127.0.0.1:48024)
0928 16:42:53.314091 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.314113 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:53.314237 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:53.315213 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:53.315217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:53.315662 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:53.315872 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.315885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.317294 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.317298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.318656 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.318671 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.319451 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.319454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.319596 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.320193 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:53.320216 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:53.322878 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:53.326178 (+ 3300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.326184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.326189 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.326554 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.326908 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.326913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.326917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.327054 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.327588 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:53.327591 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:53.327739 (+ 148us) client_negotiation.cc:769] Sending connection context
0928 16:42:53.327907 (+ 168us) client_negotiation.cc:240] Negotiation successful
0928 16:42:53.328280 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 16:42:53.328765 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.313853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48024 (local address 127.10.247.253:33955)
0928 16:42:53.314882 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.314887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.314905 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:42:53.314967 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.314972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.315053 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.315158 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.316056 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.317170 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.318803 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.319313 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.319436 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.323070 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.323076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.323079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.323114 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.326044 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.326668 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.326674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.326676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.326806 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.327148 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.327152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.327155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.327471 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.327652 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.328042 (+ 390us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.328273 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":763,"threads_started":1}
I20260928 16:42:53.330000 1490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:42:53.330370 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.314402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.10.247.254:41425)
0928 16:42:53.314580 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.314585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.314658 (+ 73us) server_negotiation.cc:407] Connection header received
0928 16:42:53.314841 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.314847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.314948 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.315048 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.316499 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.317469 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.319123 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.319694 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.319785 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.324525 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.324531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.324534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.324570 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.327245 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.327967 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.327974 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.327978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.328128 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.328564 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.328569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.328573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.329043 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.329328 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.329627 (+ 299us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.329883 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Sep 28 16:42:53 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613773, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
Sep 28 16:42:53 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
I20260928 16:42:53.344355 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.330930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47458)
0928 16:42:53.331204 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.331224 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:53.331339 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:53.331665 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:53.331670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:53.332088 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:53.332277 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.332289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.333422 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.333428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.334567 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.334582 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.335399 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.335403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.335610 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.336138 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:53.336155 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:53.338890 (+ 2735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:53.341950 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.341955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.341957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.342312 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.342707 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.342711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.342714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.342844 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.343448 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:53.343453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:53.343656 (+ 203us) client_negotiation.cc:769] Sending connection context
0928 16:42:53.343773 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 16:42:53.344030 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":2}
I20260928 16:42:53.344805 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.330904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47458 (local address 127.10.247.252:37565)
0928 16:42:53.331070 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.331074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.331284 (+ 210us) server_negotiation.cc:407] Connection header received
0928 16:42:53.331457 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.331462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.331533 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.331613 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.332417 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.333296 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.334699 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.335265 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.335395 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.339079 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.339084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.339089 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.339120 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.341824 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.342421 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.342425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.342427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.342597 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.342943 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.342948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.342951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.343326 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.343466 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.343894 (+ 428us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.344195 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":44}
I20260928 16:42:53.347872 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.332151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.10.247.253:33955)
0928 16:42:53.332322 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.332326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.332345 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:42:53.332510 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.332516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.332595 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.332711 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.334071 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.334860 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.336236 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.336878 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.337000 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.341758 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.341765 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.341769 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.341811 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.345128 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.345823 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.345827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.345831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.345971 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.346367 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.346373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.346375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.346774 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.347040 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.347268 (+ 228us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.347521 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:42:53.351454 1490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c: Bootstrap starting.
I20260928 16:42:53.353251 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:53.356180 1490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:53.357818 1490 log.cc:824] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:53.357286 1314 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5f71d0ef906644f78afd9fa2069bad77"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54168
I20260928 16:42:53.361960 1490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c: No bootstrap required, opened a new log
I20260928 16:42:53.374256 1490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } }
I20260928 16:42:53.374837 1490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dff920963e647b3a3e443a79ca08f5c, State: Initialized, Role: FOLLOWER
I20260928 16:42:53.375499 1490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } }
I20260928 16:42:53.376727 1498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.377089 1498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:53.378357 1490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [sys.catalog]: configured and running, proceeding with master startup.
W20260928 16:42:53.389595 1509 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:42:53.389766 1509 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:42:53.387013 1422 master_runner.cc:186] Error getting master registration for 127.10.247.254:41425: OK, instance_id { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" instance_seqno: 1790613772069173 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:42:53.396561 1422 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:42:53.399637 1413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 5
Sep 28 16:42:53 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613772, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
I20260928 16:42:53.417832 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.400418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47460 (local address 127.10.247.252:37565)
0928 16:42:53.400629 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.400636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.400756 (+ 120us) server_negotiation.cc:407] Connection header received
0928 16:42:53.400969 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.400976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.401071 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.401189 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.402286 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.403530 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.405347 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.406087 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.406224 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.410795 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.410800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.410805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.410844 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.414562 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.415319 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.415326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.415330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.415497 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.415941 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.415947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.415950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.416515 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.416716 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.417144 (+ 428us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.417403 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 16:42:53.417820 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.400353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47460)
0928 16:42:53.400645 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.400669 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:53.400837 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:53.401242 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:53.401248 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:53.401857 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:53.402120 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.402135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.403707 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.403714 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.405193 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.405212 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.406271 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.406278 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.406545 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.407343 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:53.407370 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:53.410576 (+ 3206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:53.414734 (+ 4158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.414740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.414746 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.415173 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.415616 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.415622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.415626 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.415815 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.416656 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:53.416661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:53.416860 (+ 199us) client_negotiation.cc:769] Sending connection context
0928 16:42:53.417001 (+ 141us) client_negotiation.cc:240] Negotiation successful
0928 16:42:53.417381 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260928 16:42:53.424711 1413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152: Bootstrap starting.
I20260928 16:42:53.429551 1413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:53.430867 1413 log.cc:824] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:53.434767 1413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152: No bootstrap required, opened a new log
I20260928 16:42:53.442728 1413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } }
I20260928 16:42:53.443100 1413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43e3ea1379854d6e9601f226c92c7152, State: Initialized, Role: FOLLOWER
I20260928 16:42:53.443634 1413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } }
I20260928 16:42:53.444916 1512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.445442 1512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:53.446237 1413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:53.454349 1346 master_runner.cc:186] Error getting master registration for 127.10.247.254:41425: OK, instance_id { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" instance_seqno: 1790613772069173 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:42:53.467247 1346 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260928 16:42:53.468091 1525 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:42:53.468232 1525 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:42:53.500594 1339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } attempt: 5
W20260928 16:42:53.507062 1444 tablet.cc:2369] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 16:42:53 dist-test-slave-9fn6 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613772, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
I20260928 16:42:53.519431 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.501612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:37565 (local address 127.0.0.1:47476)
0928 16:42:53.501960 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:53.501981 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:53.502160 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:53.502593 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:53.502598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:53.503215 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:53.503464 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.503478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.504666 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.504672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.506224 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:53.506242 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.508337 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.508343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:53.508599 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.509386 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:53.509416 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:53.512541 (+ 3125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:53.516156 (+ 3615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.516162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.516167 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.516781 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.517264 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:53.517268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:53.517270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:53.517437 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:53.518232 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:53.518237 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:53.518452 (+ 215us) client_negotiation.cc:769] Sending connection context
0928 16:42:53.518595 (+ 143us) client_negotiation.cc:240] Negotiation successful
0928 16:42:53.518962 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260928 16:42:53.520697 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.501697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.10.247.252:37565)
0928 16:42:53.501922 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.501929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.502068 (+ 139us) server_negotiation.cc:407] Connection header received
0928 16:42:53.502310 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.502317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.502423 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.502512 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.503660 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.504480 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.506377 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.506993 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.507129 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.512834 (+ 5705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.512840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.512844 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.512888 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.515980 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.516945 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.516952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.516956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.517150 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.517572 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.517578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.517582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.517995 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.518162 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.519918 (+ 1756us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.520257 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:42:53.527865 1339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77: Bootstrap starting.
I20260928 16:42:53.533016 1339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:53.534614 1339 log.cc:824] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:53.538741 1339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77: No bootstrap required, opened a new log
I20260928 16:42:53.547060 1339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } }
I20260928 16:42:53.547574 1339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f71d0ef906644f78afd9fa2069bad77, State: Initialized, Role: FOLLOWER
I20260928 16:42:53.548195 1339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } }
I20260928 16:42:53.550061 1528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.550873 1528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:53.551476 1339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:53.551535 1314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 16:42:53.551771 1314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:53.555723 1314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } }
W20260928 16:42:53.557667 1293 tablet.cc:2369] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:53.558902 1314 leader_election.cc:288] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [CANDIDATE]: Term 1 election: Requested vote from peers 43e3ea1379854d6e9601f226c92c7152 (127.10.247.253:33955), 6dff920963e647b3a3e443a79ca08f5c (127.10.247.252:37565)
I20260928 16:42:53.559126 1388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f71d0ef906644f78afd9fa2069bad77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "43e3ea1379854d6e9601f226c92c7152"
I20260928 16:42:53.559726 1388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:53.563585 1388 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f71d0ef906644f78afd9fa2069bad77 in term 1.
I20260928 16:42:53.565176 1465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f71d0ef906644f78afd9fa2069bad77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6dff920963e647b3a3e443a79ca08f5c"
I20260928 16:42:53.565770 1465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:53.568962 1289 leader_election.cc:302] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43e3ea1379854d6e9601f226c92c7152, 5f71d0ef906644f78afd9fa2069bad77; no voters:
I20260928 16:42:53.569608 1528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:53.571938 1465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f71d0ef906644f78afd9fa2069bad77 in term 1.
I20260928 16:42:53.576959 1528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [term 1 LEADER]: Becoming Leader. State: Replica: 5f71d0ef906644f78afd9fa2069bad77, State: Running, Role: LEADER
I20260928 16:42:53.577836 1528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } }
I20260928 16:42:53.586654 1531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f71d0ef906644f78afd9fa2069bad77. Latest consensus state: current_term: 1 leader_uuid: "5f71d0ef906644f78afd9fa2069bad77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.587261 1531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:53.602965 1537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:53.633740 1537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260928 16:42:53.647300 1367 tablet.cc:2369] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:53.661950 1465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [term 1 FOLLOWER]: Refusing update from remote peer 5f71d0ef906644f78afd9fa2069bad77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:53.663470 1531 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:53.673238 1388 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [term 1 FOLLOWER]: Refusing update from remote peer 5f71d0ef906644f78afd9fa2069bad77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:53.674633 1531 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:53.688714 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.598755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.10.247.253:33955)
0928 16:42:53.598973 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.598981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.621897 (+ 22916us) server_negotiation.cc:407] Connection header received
0928 16:42:53.626660 (+ 4763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.626669 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.626789 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.626872 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.634875 (+ 8003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.636186 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.652192 (+ 16006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.652946 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.653117 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.666855 (+ 13738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.666866 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.666871 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.666937 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.670857 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.681579 (+ 10722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.681587 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.681591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.681796 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.683050 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.683056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.683060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.683563 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.683879 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.687900 (+ 4021us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.688270 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 16:42:53.691717 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.605613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.10.247.252:37565)
0928 16:42:53.605896 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.605904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.623207 (+ 17303us) server_negotiation.cc:407] Connection header received
0928 16:42:53.623712 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.623720 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.623839 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.623925 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.638283 (+ 14358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.639415 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.653553 (+ 14138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.654557 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.654736 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.671926 (+ 17190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.671939 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.671944 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.672006 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.675711 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.682355 (+ 6644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.682363 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.682367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.682555 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.685155 (+ 2600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.685162 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.685166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.685645 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.685950 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.690935 (+ 4985us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.691293 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260928 16:42:53.696082 1553 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 16:42:53.697494 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:53.618235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54170 (local address 127.10.247.254:41425)
0928 16:42:53.618522 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.618530 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:53.631029 (+ 12499us) server_negotiation.cc:407] Connection header received
0928 16:42:53.631334 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:53.631341 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:53.631438 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:53.631530 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:53.643011 (+ 11481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.644656 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.657160 (+ 12504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:53.657883 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:53.658012 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:53.676887 (+ 18875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:53.676896 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:53.676901 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:53.676962 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:53.680468 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.684441 (+ 3973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.684449 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.684453 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.684668 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:53.686510 (+ 1842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:53.686518 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:53.686522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:53.686976 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:53.687279 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:53.696752 (+ 9473us) server_negotiation.cc:299] Negotiation successful
0928 16:42:53.697100 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260928 16:42:53.705222 1498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f71d0ef906644f78afd9fa2069bad77. Latest consensus state: current_term: 1 leader_uuid: "5f71d0ef906644f78afd9fa2069bad77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.705646 1498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:53.709126 1498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f71d0ef906644f78afd9fa2069bad77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.708832 1546 mvcc.cc:205] Tried to move back new op lower bound from 7334354016900452352 to 7334354016583053312. Current Snapshot: MvccSnapshot[applied={T|T < 7334354016900452352}]
I20260928 16:42:53.709406 1498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:53.717337 1512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f71d0ef906644f78afd9fa2069bad77. Latest consensus state: current_term: 1 leader_uuid: "5f71d0ef906644f78afd9fa2069bad77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.717670 1512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:53.721199 1531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f71d0ef906644f78afd9fa2069bad77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.721575 1531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:53.723294 1528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f71d0ef906644f78afd9fa2069bad77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.723630 1528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77 [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:53.725917 1551 mvcc.cc:205] Tried to move back new op lower bound from 7334354016900452352 to 7334354016583053312. Current Snapshot: MvccSnapshot[applied={T|T < 7334354016900452352 or (T in {7334354016900452352})}]
I20260928 16:42:53.728494 1537 catalog_manager.cc:1347] Generated new cluster ID: 475d6df3adfc478a8ee5bf7b82c2ff44
I20260928 16:42:53.729009 1537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:53.748064 1512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f71d0ef906644f78afd9fa2069bad77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f71d0ef906644f78afd9fa2069bad77" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 41425 } } peers { permanent_uuid: "43e3ea1379854d6e9601f226c92c7152" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 33955 } } peers { permanent_uuid: "6dff920963e647b3a3e443a79ca08f5c" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 37565 } } }
I20260928 16:42:53.748425 1512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:53.759932 1550 mvcc.cc:205] Tried to move back new op lower bound from 7334354016900452352 to 7334354016583053312. Current Snapshot: MvccSnapshot[applied={T|T < 7334354016900452352}]
I20260928 16:42:53.781126 1537 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:53.783607 1537 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:53.810884 1537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f71d0ef906644f78afd9fa2069bad77: Generated new TSK 0
I20260928 16:42:53.812637 1537 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 16:42:53.942163 11231 mini_kdc.cc:348] Time spent kdestroy: real 0.104s user 0.000s sys 0.033s
W20260928 16:42:54.004254 1493 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:53.991295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47494 (local address 127.10.247.252:37565)
0928 16:42:53.991865 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.991872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.000890 (+ 9018us) server_negotiation.cc:407] Connection header received
0928 16:42:54.002206 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.002214 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.002331 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.002425 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.003930 (+ 1505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47494: BlockingRecv error: recv got EOF from 127.0.0.1:47494 (error 108)
Metrics: {"server-negotiator.queue_time_us":380}
W20260928 16:42:54.007202 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:53.991573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.10.247.253:33955)
0928 16:42:53.992097 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.992103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.004652 (+ 12549us) server_negotiation.cc:407] Connection header received
0928 16:42:54.005267 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.005278 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.005393 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.005482 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.006878 (+ 1396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48044: BlockingRecv error: recv got EOF from 127.0.0.1:48044 (error 108)
Metrics: {"server-negotiator.queue_time_us":332}
W20260928 16:42:54.009625 1494 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:53.998653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.10.247.254:41425)
0928 16:42:53.998943 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:53.998951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.006336 (+ 7385us) server_negotiation.cc:407] Connection header received
0928 16:42:54.007151 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.007159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.007266 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.007349 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.009298 (+ 1949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54176: BlockingRecv error: recv got EOF from 127.0.0.1:54176 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260928 16:42:54.018254 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:54.015942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.10.247.253:33955)
0928 16:42:54.016169 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.016175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.016304 (+ 129us) server_negotiation.cc:407] Connection header received
0928 16:42:54.016733 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.016739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.016885 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.016987 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.017966 (+ 979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48060: BlockingRecv error: recv got EOF from 127.0.0.1:48060 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260928 16:42:54.035692 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:54.028214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.10.247.253:33955)
0928 16:42:54.028464 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.028471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.033427 (+ 4956us) server_negotiation.cc:407] Connection header received
0928 16:42:54.033751 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.033759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.033883 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.033999 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.035286 (+ 1287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48068: BlockingRecv error: recv got EOF from 127.0.0.1:48068 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260928 16:42:54.047190 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:54.043669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.10.247.253:33955)
0928 16:42:54.043906 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.043918 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.045271 (+ 1353us) server_negotiation.cc:407] Connection header received
0928 16:42:54.045504 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.045510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.045629 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.045753 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.046808 (+ 1055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48082: BlockingRecv error: recv got EOF from 127.0.0.1:48082 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260928 16:42:54.063779 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:54.059453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.10.247.253:33955)
0928 16:42:54.059764 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.059771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.061820 (+ 2049us) server_negotiation.cc:407] Connection header received
0928 16:42:54.062102 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.062109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.062239 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.062375 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.063478 (+ 1103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48096: BlockingRecv error: recv got EOF from 127.0.0.1:48096 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
W20260928 16:42:54.091562 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:54.079166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48108 (local address 127.10.247.253:33955)
0928 16:42:54.079405 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.079412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.089401 (+ 9989us) server_negotiation.cc:407] Connection header received
0928 16:42:54.089760 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.089771 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.089898 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.089977 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.091245 (+ 1268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48108: BlockingRecv error: recv got EOF from 127.0.0.1:48108 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260928 16:42:54.115160 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:54.112644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48116 (local address 127.10.247.253:33955)
0928 16:42:54.112904 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.112912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.113525 (+ 613us) server_negotiation.cc:407] Connection header received
0928 16:42:54.113752 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.113757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.113854 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.113954 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.114854 (+ 900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48116: BlockingRecv error: recv got EOF from 127.0.0.1:48116 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260928 16:42:54.154560 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:54.151857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.10.247.253:33955)
0928 16:42:54.152103 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.152112 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.152836 (+ 724us) server_negotiation.cc:407] Connection header received
0928 16:42:54.153089 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.153094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.153213 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.153315 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.154253 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48118: BlockingRecv error: recv got EOF from 127.0.0.1:48118 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260928 16:42:54.226359 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:54.223901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48134 (local address 127.10.247.253:33955)
0928 16:42:54.224144 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.224153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.224553 (+ 400us) server_negotiation.cc:407] Connection header received
0928 16:42:54.224812 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.224819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.224951 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.225091 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.226039 (+ 948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48134: BlockingRecv error: recv got EOF from 127.0.0.1:48134 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260928 16:42:54.369230 1496 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:54.366608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48146 (local address 127.10.247.253:33955)
0928 16:42:54.366820 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.366826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.367228 (+ 402us) server_negotiation.cc:407] Connection header received
0928 16:42:54.367512 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.367519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.367652 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.367781 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:54.368937 (+ 1156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48146: BlockingRecv error: recv got EOF from 127.0.0.1:48146 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 16:42:54.392373 1509 catalog_manager.cc:1259] Loaded cluster ID: 475d6df3adfc478a8ee5bf7b82c2ff44
I20260928 16:42:54.392490 1509 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c: loading cluster ID for follower catalog manager: success
I20260928 16:42:54.395382 1509 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c: acquiring CA information for follower catalog manager: success
I20260928 16:42:54.397809 1509 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6dff920963e647b3a3e443a79ca08f5c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:42:54.470942 1525 catalog_manager.cc:1259] Loaded cluster ID: 475d6df3adfc478a8ee5bf7b82c2ff44
I20260928 16:42:54.471076 1525 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152: loading cluster ID for follower catalog manager: success
I20260928 16:42:54.474032 1525 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152: acquiring CA information for follower catalog manager: success
I20260928 16:42:54.476285 1525 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 43e3ea1379854d6e9601f226c92c7152: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:42:54.637189 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:54.630528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48150 (local address 127.10.247.253:33955)
0928 16:42:54.630807 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.630815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.631079 (+ 264us) server_negotiation.cc:407] Connection header received
0928 16:42:54.631339 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.631345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.631568 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.631692 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:54.632222 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.633079 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.634608 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.635038 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.635202 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:54.635750 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:54.636462 (+ 712us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:54.636675 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:54.636766 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 16:42:54.636873 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:42:54.650130 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:54.643848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.10.247.252:37565)
0928 16:42:54.644298 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.644303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.644329 (+ 26us) server_negotiation.cc:407] Connection header received
0928 16:42:54.644557 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.644561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.644810 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.644968 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:54.645328 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.646093 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.647592 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.648029 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.648130 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:54.648745 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:54.649496 (+ 751us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:54.649623 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:54.649722 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 16:42:54.649811 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":245,"threads_started":1}
I20260928 16:42:54.667058 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:54.658638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.10.247.252:37565)
0928 16:42:54.658855 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.658861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.661028 (+ 2167us) server_negotiation.cc:407] Connection header received
0928 16:42:54.661116 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.661120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.661213 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.661295 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:54.661814 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.662800 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.664947 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.665584 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.665713 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:54.666274 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:54.666442 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:54.666536 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:54.666655 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 16:42:54.666750 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 16:42:54.667094 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:54.658180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48162 (local address 127.10.247.253:33955)
0928 16:42:54.658441 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.658449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.660337 (+ 1888us) server_negotiation.cc:407] Connection header received
0928 16:42:54.660578 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.660585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.660749 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.660862 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:54.661523 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.662606 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.664977 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.665476 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.665552 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:54.666298 (+ 746us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:54.666414 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:54.666463 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:54.666692 (+ 229us) server_negotiation.cc:299] Negotiation successful
0928 16:42:54.666789 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260928 16:42:54.670356 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:54.657822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.10.247.254:41425)
0928 16:42:54.659863 (+ 2041us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:54.659869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:54.661542 (+ 1673us) server_negotiation.cc:407] Connection header received
0928 16:42:54.661739 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:54.661745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:54.662014 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:54.662219 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:54.662813 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.663583 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.667753 (+ 4170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:54.668440 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:54.668561 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:54.669286 (+ 725us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:54.669743 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:54.669827 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:54.669961 (+ 134us) server_negotiation.cc:299] Negotiation successful
0928 16:42:54.670046 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1901,"thread_start_us":1784,"threads_started":1}
I20260928 16:42:54.675318 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1272
I20260928 16:42:54.677732 1334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:54.802346 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 1272
W20260928 16:42:54.823609 1439 connection.cc:535] server connection from 127.0.0.1:47476 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 16:42:54.824368 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1346
I20260928 16:42:54.825753 1408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:54.947758 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 1346
I20260928 16:42:54.968323 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1422
I20260928 16:42:54.969749 1485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:55.092203 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 1422
2026-09-28T16:42:55Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4924 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:42:55 dist-test-slave-9fn6 krb5kdc[1638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:42:55 dist-test-slave-9fn6 krb5kdc[1638](info): set up 2 sockets
Sep 28 16:42:55 dist-test-slave-9fn6 krb5kdc[1638](info): commencing operation
krb5kdc: starting...
W20260928 16:42:56.158252 11231 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.006s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/test-user.keytab.
Sep 28 16:42:56 dist-test-slave-9fn6 krb5kdc[1638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:42:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:42:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.247.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
Entry for principal kudu/127.10.247.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab.
I20260928 16:42:56.488477 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44475
--webserver_interface=127.10.247.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:44475,127.10.247.253:46561,127.10.247.252:33909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:42:56.877928 1651 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:42:56.878376 1651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:56.878485 1651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:56.878604 1651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:56.886848 1651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:56.886986 1651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:56.887058 1651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:56.887135 1651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:56.887241 1651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:56.899564 1651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:44475,127.10.247.253:46561,127.10.247.252:33909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.254.keytab
--principal=kudu/127.10.247.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.254:44475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.247.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:56.901523 1651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:56.903201 1651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:56.916760 1657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:56.916795 1659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:56.916821 1656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:56.918078 1651 server_base.cc:1034] running on GCE node
I20260928 16:42:56.918991 1651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:56.920845 1651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:56.922115 1651 hybrid_clock.cc:648] HybridClock initialized: now 1790613776922019 us; error 44 us; skew 500 ppm
I20260928 16:42:56.922497 1651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:56 dist-test-slave-9fn6 krb5kdc[1638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613776, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:56.926115 1651 init.cc:377] Logged in from keytab as kudu/127.10.247.254@KRBTEST.COM (short username kudu)
I20260928 16:42:56.927947 1651 webserver.cc:460] Webserver started at http://127.10.247.254:42331/ using document root <none> and password file <none>
I20260928 16:42:56.928867 1651 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:56.928970 1651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:56.929350 1651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:56.933097 1651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/data/instance:
uuid: "8394c5357c354b21b27053b3de7fa16f"
format_stamp: "Formatted at 2026-09-28 16:42:56 on dist-test-slave-9fn6"
server_key: "363b0f6077fae6830e3614a916a7ee09"
server_key_iv: "4c253cdf5d2deaebe285705b564a4825"
server_key_version: "encryptionkey@0"
I20260928 16:42:56.934072 1651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/wal/instance:
uuid: "8394c5357c354b21b27053b3de7fa16f"
format_stamp: "Formatted at 2026-09-28 16:42:56 on dist-test-slave-9fn6"
server_key: "363b0f6077fae6830e3614a916a7ee09"
server_key_iv: "4c253cdf5d2deaebe285705b564a4825"
server_key_version: "encryptionkey@0"
I20260928 16:42:56.940591 1651 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260928 16:42:56.944958 1666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:56.946645 1651 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:42:56.946841 1651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/wal
uuid: "8394c5357c354b21b27053b3de7fa16f"
format_stamp: "Formatted at 2026-09-28 16:42:56 on dist-test-slave-9fn6"
server_key: "363b0f6077fae6830e3614a916a7ee09"
server_key_iv: "4c253cdf5d2deaebe285705b564a4825"
server_key_version: "encryptionkey@0"
I20260928 16:42:56.947119 1651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:56.973821 1651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:56.979279 1651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:56.979621 1651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:57.000679 1651 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.254:44475
I20260928 16:42:57.000690 1718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.254:44475 every 8 connection(s)
I20260928 16:42:57.002493 1651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/data/info.pb
I20260928 16:42:57.003005 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 1651
I20260928 16:42:57.003202 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-0/wal/instance
I20260928 16:42:57.003786 11231 external_mini_cluster.cc:1428] Setting key 1c11254a5dd0cca9241c3e833c8dc423
I20260928 16:42:57.007606 1719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:57.015952 1719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:42:57 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613776, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.247.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.253@KRBTEST.COM" created.
I20260928 16:42:57.043500 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.020318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44475 (local address 127.0.0.1:44906)
0928 16:42:57.021373 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.021427 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:57.023622 (+ 2195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:57.025150 (+ 1528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:57.025158 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:57.025990 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:57.026395 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:57.026414 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.028083 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.028088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:57.029312 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:57.029326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.030329 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.030336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:57.030522 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:57.031315 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:57.031344 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:57.034244 (+ 2900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:57.038229 (+ 3985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:57.038244 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:57.038268 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:57.038735 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:57.039208 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:57.039213 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:57.039216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:57.039380 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:57.040364 (+ 984us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:57.040374 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:57.040657 (+ 283us) client_negotiation.cc:769] Sending connection context
0928 16:42:57.040942 (+ 285us) client_negotiation.cc:240] Negotiation successful
0928 16:42:57.041731 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":179,"thread_start_us":463,"threads_started":1}
I20260928 16:42:57.043526 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.020318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44906 (local address 127.10.247.254:44475)
0928 16:42:57.024359 (+ 4041us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:57.024365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:57.024398 (+ 33us) server_negotiation.cc:407] Connection header received
0928 16:42:57.024564 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:57.024570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:57.024906 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:57.025115 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:57.026612 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.027934 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.029451 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.030082 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.030286 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:57.034565 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:57.034576 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:57.034593 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:57.034639 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:57.038057 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:57.038897 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:57.038905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:57.038908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:57.039092 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:57.039494 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:57.039505 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:57.039507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:57.040211 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:57.040435 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:57.041099 (+ 664us) server_negotiation.cc:299] Negotiation successful
0928 16:42:57.041587 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":3846,"spinlock_wait_cycles":90368,"thread_start_us":3611,"threads_started":1}
I20260928 16:42:57.047379 1719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:42:57.048840 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.048174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:53988)
0928 16:42:57.048415 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.048591 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260928 16:42:57.049191 1669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
I20260928 16:42:57.051020 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.050599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:53998)
0928 16:42:57.050766 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.050850 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260928 16:42:57.051294 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:46561: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab.
Entry for principal kudu/127.10.247.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab.
I20260928 16:42:57.076735 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.253:46561
--webserver_interface=127.10.247.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab
--principal=kudu/127.10.247.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:44475,127.10.247.253:46561,127.10.247.252:33909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:42:57.100785 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } attempt: 1
I20260928 16:42:57.102550 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.101839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54014)
0928 16:42:57.102118 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.102309 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260928 16:42:57.103888 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.103393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54030)
0928 16:42:57.103571 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.103690 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 16:42:57.104106 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:46561: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
I20260928 16:42:57.172464 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } attempt: 2
I20260928 16:42:57.174291 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.173539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54040)
0928 16:42:57.173886 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.174051 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260928 16:42:57.175599 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.175060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54048)
0928 16:42:57.175248 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.175373 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260928 16:42:57.175825 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:46561: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
I20260928 16:42:57.267164 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } attempt: 3
I20260928 16:42:57.268867 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.268173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54056)
0928 16:42:57.268446 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.268602 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260928 16:42:57.270262 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.269730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54064)
0928 16:42:57.269909 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.270025 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260928 16:42:57.270514 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:46561: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
I20260928 16:42:57.413829 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } attempt: 4
I20260928 16:42:57.415553 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.414758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54076)
0928 16:42:57.415156 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.415294 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260928 16:42:57.416960 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.416404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54084)
0928 16:42:57.416595 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.416747 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260928 16:42:57.417200 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.253:46561: Network error: Client connection negotiation failed: client connection to 127.10.247.253:46561: connect: Connection refused (error 111)
W20260928 16:42:57.474802 1726 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:42:57.475224 1726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:57.475304 1726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:57.475396 1726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:57.483314 1726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:57.483425 1726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:57.483486 1726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:57.483539 1726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:57.483606 1726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:57.495087 1726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:44475,127.10.247.253:46561,127.10.247.252:33909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.253.keytab
--principal=kudu/127.10.247.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.253:46561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.247.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:57.496886 1726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:57.498504 1726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:57.511548 1735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:57.511633 1732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:57.511586 1731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:57.512930 1726 server_base.cc:1034] running on GCE node
I20260928 16:42:57.513839 1726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:57.515614 1726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:57.516865 1726 hybrid_clock.cc:648] HybridClock initialized: now 1790613777516784 us; error 44 us; skew 500 ppm
I20260928 16:42:57.517271 1726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:57 dist-test-slave-9fn6 krb5kdc[1638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613777, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:57.521080 1726 init.cc:377] Logged in from keytab as kudu/127.10.247.253@KRBTEST.COM (short username kudu)
I20260928 16:42:57.522889 1726 webserver.cc:460] Webserver started at http://127.10.247.253:35953/ using document root <none> and password file <none>
I20260928 16:42:57.523787 1726 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:57.523906 1726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:57.524276 1726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:57.528071 1726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/data/instance:
uuid: "158e1da36b1241ed9353dbcbc3346ebf"
format_stamp: "Formatted at 2026-09-28 16:42:57 on dist-test-slave-9fn6"
server_key: "191eb276713ff2bfe27eb4695b8a546c"
server_key_iv: "cccbe357dbc97056252020b25d26c0a5"
server_key_version: "encryptionkey@0"
I20260928 16:42:57.529054 1726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/wal/instance:
uuid: "158e1da36b1241ed9353dbcbc3346ebf"
format_stamp: "Formatted at 2026-09-28 16:42:57 on dist-test-slave-9fn6"
server_key: "191eb276713ff2bfe27eb4695b8a546c"
server_key_iv: "cccbe357dbc97056252020b25d26c0a5"
server_key_version: "encryptionkey@0"
I20260928 16:42:57.535444 1726 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260928 16:42:57.539705 1742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:57.541392 1726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:42:57.541584 1726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/wal
uuid: "158e1da36b1241ed9353dbcbc3346ebf"
format_stamp: "Formatted at 2026-09-28 16:42:57 on dist-test-slave-9fn6"
server_key: "191eb276713ff2bfe27eb4695b8a546c"
server_key_iv: "cccbe357dbc97056252020b25d26c0a5"
server_key_version: "encryptionkey@0"
I20260928 16:42:57.541841 1726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:57.564749 1726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:57.569840 1726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:57.570164 1726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:57.590575 1726 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.253:46561
I20260928 16:42:57.590605 1793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.253:46561 every 8 connection(s)
I20260928 16:42:57.592291 1726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/data/info.pb
I20260928 16:42:57.597144 1794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:57.601289 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 1726
I20260928 16:42:57.601502 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-1/wal/instance
I20260928 16:42:57.601987 11231 external_mini_cluster.cc:1428] Setting key 3334985c5b15d895c8549e4371a07e46
I20260928 16:42:57.605082 1794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:42:57 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613777, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:57.630422 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.608172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44908 (local address 127.10.247.254:44475)
0928 16:42:57.608878 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:57.608885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:57.609959 (+ 1074us) server_negotiation.cc:407] Connection header received
0928 16:42:57.612091 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:57.612099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:57.612202 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:57.612348 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:57.614878 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.615764 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.617999 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.618452 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.618553 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:57.623068 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:57.623074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:57.623077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:57.623125 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:57.626074 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:57.627223 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:57.627229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:57.627232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:57.627416 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:57.627992 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:57.627997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:57.627999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:57.628418 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:57.628591 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:57.629833 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0928 16:42:57.630086 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":310,"threads_started":1}
I20260928 16:42:57.632285 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.608866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44475 (local address 127.0.0.1:44908)
0928 16:42:57.609766 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.609822 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:57.611792 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:57.612760 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:57.612774 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:57.613663 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:57.614594 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:57.614621 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.615942 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.615948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:57.617850 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:57.617867 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.618689 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.618695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:57.619006 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:57.619838 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:57.619869 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:57.622768 (+ 2899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:57.626255 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:57.626266 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:57.626304 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:57.627025 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:57.627569 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:57.627574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:57.627577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:57.627850 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:57.628581 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:57.628591 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:57.629294 (+ 703us) client_negotiation.cc:769] Sending connection context
0928 16:42:57.629691 (+ 397us) client_negotiation.cc:240] Negotiation successful
0928 16:42:57.630393 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":366,"threads_started":1}
I20260928 16:42:57.634697 1794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.247.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.247.252@KRBTEST.COM" created.
Sep 28 16:42:57 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613777, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
I20260928 16:42:57.652184 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.635610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54092)
0928 16:42:57.635802 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.635819 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:57.635924 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:57.637176 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:57.637180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:57.637626 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:57.637842 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:57.637856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.639658 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.639663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:57.640954 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:57.640972 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.641866 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.641871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:57.642048 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:57.642695 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:57.642718 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:57.645288 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:57.649148 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:57.649154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:57.649159 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:57.649624 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:57.650069 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:57.650074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:57.650079 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:57.650242 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:57.650980 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:57.650990 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:57.651152 (+ 162us) client_negotiation.cc:769] Sending connection context
0928 16:42:57.651294 (+ 142us) client_negotiation.cc:240] Negotiation successful
0928 16:42:57.651648 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260928 16:42:57.652247 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.635871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.10.247.253:46561)
0928 16:42:57.636573 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:57.636581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:57.636644 (+ 63us) server_negotiation.cc:407] Connection header received
0928 16:42:57.636722 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:57.636729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:57.636974 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:57.637133 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:57.638068 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.639499 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.641097 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.641715 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.641814 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:57.645548 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:57.645563 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:57.645586 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:57.645629 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:57.649003 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:57.649776 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:57.649784 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:57.649786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:57.649928 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:57.650354 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:57.650360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:57.650364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:57.650838 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:57.651078 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:57.651491 (+ 413us) server_negotiation.cc:299] Negotiation successful
0928 16:42:57.651753 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":356,"threads_started":1}
I20260928 16:42:57.655411 1794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:42:57.657004 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.656393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37658)
0928 16:42:57.656647 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.656809 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260928 16:42:57.657392 1744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
I20260928 16:42:57.659122 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.658745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37660)
0928 16:42:57.658913 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.658999 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 16:42:57.659365 1794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.247.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab.
Entry for principal kudu/127.10.247.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab.
I20260928 16:42:57.680634 11231 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKneNwT/build/asan/bin/kudu
/tmp/dist-test-taskKneNwT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.252:33909
--webserver_interface=127.10.247.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab
--principal=kudu/127.10.247.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.247.212:41959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.247.254:44475,127.10.247.253:46561,127.10.247.252:33909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:42:57.708770 1794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 1
I20260928 16:42:57.710274 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.709728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37670)
0928 16:42:57.709966 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.710070 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260928 16:42:57.711493 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.711089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37678)
0928 16:42:57.711266 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.711357 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 16:42:57.711707 1794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
I20260928 16:42:57.716496 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } attempt: 5
Sep 28 16:42:57 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613776, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
I20260928 16:42:57.732249 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.717659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.10.247.253:46561)
0928 16:42:57.717829 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:57.717835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:57.717856 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:42:57.717952 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:57.717956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:57.718046 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:57.718147 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:57.719202 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.719987 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.721792 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.722293 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.722398 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:57.726543 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:57.726549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:57.726552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:57.726595 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:57.729442 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:57.730308 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:57.730312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:57.730314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:57.730463 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:57.730874 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:57.730877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:57.730879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:57.731212 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:57.731335 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:57.731736 (+ 401us) server_negotiation.cc:299] Negotiation successful
0928 16:42:57.731951 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 16:42:57.732448 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.717397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54096)
0928 16:42:57.717647 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.717671 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:57.717831 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:57.718190 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:57.718194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:57.718770 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:57.719030 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:57.719043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.720120 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.720126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:57.721629 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:57.721651 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:57.722467 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:57.722473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:57.722704 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:57.723433 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:57.723470 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:57.726295 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:57.729601 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:57.729606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:57.729611 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:57.730192 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:57.730579 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:57.730584 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:57.730588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:57.730773 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:57.731316 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:57.731321 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:57.731512 (+ 191us) client_negotiation.cc:769] Sending connection context
0928 16:42:57.731668 (+ 156us) client_negotiation.cc:240] Negotiation successful
0928 16:42:57.732052 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260928 16:42:57.733742 1719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:42:57.734926 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.734493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37694)
0928 16:42:57.734662 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.734765 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 16:42:57.735829 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.735512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37704)
0928 16:42:57.735652 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.735720 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 16:42:57.736027 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
I20260928 16:42:57.779954 1794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 2
I20260928 16:42:57.781486 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.780909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37720)
0928 16:42:57.781170 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.781285 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260928 16:42:57.782598 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.782212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37732)
0928 16:42:57.782371 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.782455 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260928 16:42:57.782825 1794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
I20260928 16:42:57.787302 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 1
I20260928 16:42:57.788669 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.788134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37748)
0928 16:42:57.788331 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.788444 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 16:42:57.789773 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.789403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37764)
0928 16:42:57.789555 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.789640 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260928 16:42:57.789990 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
I20260928 16:42:57.858234 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 2
I20260928 16:42:57.859666 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.859071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37776)
0928 16:42:57.859348 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.859459 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260928 16:42:57.860774 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.860376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37786)
0928 16:42:57.860529 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.860643 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260928 16:42:57.860991 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
I20260928 16:42:57.874055 1794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 3
I20260928 16:42:57.875401 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.874869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37788)
0928 16:42:57.875099 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.875205 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260928 16:42:57.876550 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.876093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37792)
0928 16:42:57.876335 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.876420 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260928 16:42:57.876803 1794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
I20260928 16:42:57.967236 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 3
I20260928 16:42:57.968665 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.968056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37802)
0928 16:42:57.968300 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.968411 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260928 16:42:57.970005 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:57.969496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37804)
0928 16:42:57.969728 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:57.969823 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260928 16:42:57.970340 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
I20260928 16:42:58.020071 1794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 4
I20260928 16:42:58.021593 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.020978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37808)
0928 16:42:58.021262 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:58.021375 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260928 16:42:58.023008 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.022554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37814)
0928 16:42:58.022751 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:58.022851 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260928 16:42:58.023283 1794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
W20260928 16:42:58.081465 1802 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:42:58.081930 1802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:42:58.082028 1802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:42:58.082141 1802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:42:58.090451 1802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:42:58.090586 1802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:42:58.090653 1802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:42:58.090709 1802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:42:58.090777 1802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:42:58.102286 1802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.247.212:41959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.247.254:44475,127.10.247.253:46561,127.10.247.252:33909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/krb5kdc/kudu_127.10.247.252.keytab
--principal=kudu/127.10.247.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.247.252:33909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.247.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 15:46:59 UTC on ccycloud.quasar-awnebh.root.comops.site
build id 4677567
ASAN enabled
I20260928 16:42:58.104136 1802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:42:58.105830 1802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:42:58.119475 1807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:58.119496 1810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:42:58.119526 1808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:42:58.120476 1802 server_base.cc:1034] running on GCE node
I20260928 16:42:58.121531 1802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:42:58.123404 1802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:42:58.124702 1802 hybrid_clock.cc:648] HybridClock initialized: now 1790613778124567 us; error 78 us; skew 500 ppm
I20260928 16:42:58.125118 1802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:42:58 dist-test-slave-9fn6 krb5kdc[1638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613778, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:42:58.129122 1802 init.cc:377] Logged in from keytab as kudu/127.10.247.252@KRBTEST.COM (short username kudu)
I20260928 16:42:58.131073 1802 webserver.cc:460] Webserver started at http://127.10.247.252:46163/ using document root <none> and password file <none>
I20260928 16:42:58.132009 1802 fs_manager.cc:359] Metadata directory not provided
I20260928 16:42:58.132117 1802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:42:58.132455 1802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:42:58.136309 1802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/data/instance:
uuid: "ef07135ca00942779ea8311e84593dd2"
format_stamp: "Formatted at 2026-09-28 16:42:58 on dist-test-slave-9fn6"
server_key: "b7d947740b4d3de79d0baba5f4497e9a"
server_key_iv: "23528b7b8701afd4514744e6c4ccbc22"
server_key_version: "encryptionkey@0"
I20260928 16:42:58.137346 1802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/wal/instance:
uuid: "ef07135ca00942779ea8311e84593dd2"
format_stamp: "Formatted at 2026-09-28 16:42:58 on dist-test-slave-9fn6"
server_key: "b7d947740b4d3de79d0baba5f4497e9a"
server_key_iv: "23528b7b8701afd4514744e6c4ccbc22"
server_key_version: "encryptionkey@0"
I20260928 16:42:58.143996 1802 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260928 16:42:58.147593 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 4
I20260928 16:42:58.148669 1817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:42:58.149132 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.148473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37828)
0928 16:42:58.148762 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:58.148904 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260928 16:42:58.150605 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.150176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37842)
0928 16:42:58.150353 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:58.150441 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260928 16:42:58.150856 1719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.247.252:33909: Network error: Client connection negotiation failed: client connection to 127.10.247.252:33909: connect: Connection refused (error 111)
I20260928 16:42:58.150949 1802 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 16:42:58.151182 1802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/data,/tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/wal
uuid: "ef07135ca00942779ea8311e84593dd2"
format_stamp: "Formatted at 2026-09-28 16:42:58 on dist-test-slave-9fn6"
server_key: "b7d947740b4d3de79d0baba5f4497e9a"
server_key_iv: "23528b7b8701afd4514744e6c4ccbc22"
server_key_version: "encryptionkey@0"
I20260928 16:42:58.151471 1802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:42:58.176199 1802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:42:58.182189 1802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:42:58.182564 1802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:42:58.205106 1802 rpc_server.cc:307] RPC server started. Bound to: 127.10.247.252:33909
I20260928 16:42:58.205153 1868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.247.252:33909 every 8 connection(s)
I20260928 16:42:58.206861 1802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/data/info.pb
I20260928 16:42:58.211676 1869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:42:58.216310 11231 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKneNwT/build/asan/bin/kudu as pid 1802
I20260928 16:42:58.216562 11231 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKneNwT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790613388455587-11231-0/minicluster-data/master-2/wal/instance
I20260928 16:42:58.217271 11231 external_mini_cluster.cc:1428] Setting key 9df36d5e216717cdb721818fde6354b0
I20260928 16:42:58.221081 1869 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:42:58 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613778, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
Sep 28 16:42:58 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
I20260928 16:42:58.244856 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.225549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44920 (local address 127.10.247.254:44475)
0928 16:42:58.226167 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.226173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.226709 (+ 536us) server_negotiation.cc:407] Connection header received
0928 16:42:58.226911 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.226915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.227004 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.227103 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.228551 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.229344 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.230832 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.231382 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.231483 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.237834 (+ 6351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.237843 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.237849 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.237915 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.240931 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.241774 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.241779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.241782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.241935 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.242392 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.242396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.242398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.242739 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.242891 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.243816 (+ 925us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.244129 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":278,"threads_started":1}
I20260928 16:42:58.245918 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.219758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37856 (local address 127.10.247.252:33909)
0928 16:42:58.220705 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.220720 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.220770 (+ 50us) server_negotiation.cc:407] Connection header received
0928 16:42:58.222554 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.222585 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.223140 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.223599 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.225473 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.227684 (+ 2211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.229159 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.229961 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.230146 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.238333 (+ 8187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.238346 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.238369 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.238412 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.241451 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.242279 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.242288 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.242290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.242432 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.242846 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.242850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.242852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.243360 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.243707 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.244045 (+ 338us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.244528 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":192,"server-negotiator.queue_time_us":662,"thread_start_us":409,"threads_started":1}
I20260928 16:42:58.245949 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.225576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.254:44475 (local address 127.0.0.1:44920)
0928 16:42:58.226528 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:58.226601 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:58.226787 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:58.227202 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:58.227210 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:58.228101 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:58.228377 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.228385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.229500 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.229504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.230677 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.230693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.231624 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.231630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.231770 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.234700 (+ 2930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:58.234728 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:58.237489 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:58.241121 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.241144 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.241161 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.241641 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.242046 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.242052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.242056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.242276 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.242870 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:58.242880 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:58.243394 (+ 514us) client_negotiation.cc:769] Sending connection context
0928 16:42:58.243692 (+ 298us) client_negotiation.cc:240] Negotiation successful
0928 16:42:58.244372 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":3,"thread_start_us":496,"threads_started":1}
I20260928 16:42:58.247812 1869 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:42:58 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613778, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
Sep 28 16:42:58 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.254@KRBTEST.COM
I20260928 16:42:58.263022 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.248587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.253:46561 (local address 127.0.0.1:54106)
0928 16:42:58.248885 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:58.248905 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:58.249024 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:58.249882 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:58.249885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:58.250266 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:58.250481 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.250491 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.251683 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.251688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.252756 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.252769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.253505 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.253511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.253738 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.254350 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:58.254370 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:58.256995 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:58.260376 (+ 3381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.260382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.260387 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.260893 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.261292 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.261298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.261301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.261447 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.261981 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:58.261987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:58.262159 (+ 172us) client_negotiation.cc:769] Sending connection context
0928 16:42:58.262277 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 16:42:58.262632 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260928 16:42:58.263111 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.248758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54106 (local address 127.10.247.253:46561)
0928 16:42:58.249354 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.249358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.249375 (+ 17us) server_negotiation.cc:407] Connection header received
0928 16:42:58.249439 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.249444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.249535 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.249669 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.250661 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.251544 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.252902 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.253353 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.253469 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.257239 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.257251 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.257254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.257297 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.260235 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.261022 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.261027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.261029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.261183 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.261555 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.261559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.261561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.261857 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.262026 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.262413 (+ 387us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.262644 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":334,"threads_started":1}
I20260928 16:42:58.264209 1869 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:42:58.264348 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.249571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.10.247.254:44475)
0928 16:42:58.249775 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.249780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.249797 (+ 17us) server_negotiation.cc:407] Connection header received
0928 16:42:58.249941 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.249946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.250020 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.250100 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.251416 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.252206 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.253561 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.254018 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.254130 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.258567 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.258573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.258576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.258614 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.261256 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.261944 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.261950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.261954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.262085 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.262440 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.262445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.262448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.262747 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.263042 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.263897 (+ 855us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.264094 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Sep 28 16:42:58 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613778, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.252@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
Sep 28 16:42:58 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.247.253@KRBTEST.COM
I20260928 16:42:58.278090 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.265030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37870)
0928 16:42:58.265283 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:58.265305 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:58.265415 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:58.265719 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:58.265723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:58.266221 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:58.266425 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.266436 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.267524 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.267530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.268563 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.268575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.269305 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.269311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.269542 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.270137 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:58.270170 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:58.272556 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:58.275673 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.275677 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.275680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.276024 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.276351 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.276356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.276360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.276482 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.277051 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:58.277055 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:58.277220 (+ 165us) client_negotiation.cc:769] Sending connection context
0928 16:42:58.277341 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 16:42:58.277655 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260928 16:42:58.278118 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.265151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37870 (local address 127.10.247.252:33909)
0928 16:42:58.265304 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.265310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.265347 (+ 37us) server_negotiation.cc:407] Connection header received
0928 16:42:58.265527 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.265531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.265601 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.265674 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.266556 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.267390 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.268720 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.269160 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.269254 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.272762 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.272768 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.272772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.272803 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.275532 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.276129 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.276134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.276137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.276253 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.276575 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.276581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.276584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.276947 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.277083 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.277437 (+ 354us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.277826 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":53}
I20260928 16:42:58.281450 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.266163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.10.247.253:46561)
0928 16:42:58.266327 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.266331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.266345 (+ 14us) server_negotiation.cc:407] Connection header received
0928 16:42:58.266490 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.266495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.266571 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.266672 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.267934 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.268681 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.270153 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.270583 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.270677 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.275355 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.275362 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.275367 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.275413 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.278120 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.279343 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.279349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.279352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.279542 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.279931 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.279936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.279939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.280305 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.280573 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.280859 (+ 286us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.281119 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:42:58.284554 1869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2: Bootstrap starting.
I20260928 16:42:58.286676 11231 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:42:58.288640 1869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:58.289903 1869 log.cc:824] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:58.289237 1694 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8394c5357c354b21b27053b3de7fa16f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44926
I20260928 16:42:58.293210 1869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2: No bootstrap required, opened a new log
I20260928 16:42:58.301158 1869 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } }
I20260928 16:42:58.301576 1869 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef07135ca00942779ea8311e84593dd2, State: Initialized, Role: FOLLOWER
I20260928 16:42:58.302131 1869 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } }
I20260928 16:42:58.303190 1878 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.303629 1878 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:58.304399 1869 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 16:42:58.315542 1889 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:42:58.315730 1889 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:42:58.313086 1802 master_runner.cc:186] Error getting master registration for 127.10.247.254:44475: OK, instance_id { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" instance_seqno: 1790613776994580 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:42:58.322201 1802 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:42:58.322520 1794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 5
Sep 28 16:42:58 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613777, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.253@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
I20260928 16:42:58.342057 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.323489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.10.247.252:33909)
0928 16:42:58.324238 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.324246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.324269 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:42:58.324349 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.324355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.324444 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.324549 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.325755 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.327004 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.328795 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.329615 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.329759 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.334415 (+ 4656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.334424 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.334429 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.334486 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.338399 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.339400 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.339407 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.339411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.339583 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.340078 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.340083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.340086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.340694 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.340876 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.341360 (+ 484us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.341640 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558}
I20260928 16:42:58.342056 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.323406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37884)
0928 16:42:58.323750 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:58.323776 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:58.323938 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:58.324669 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:58.324675 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:58.325295 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:58.325545 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.325560 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.327169 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.327175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.328641 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.328658 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.329781 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.329788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.330046 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.330834 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:58.330859 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:58.334158 (+ 3299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:58.338603 (+ 4445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.338609 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.338614 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.339256 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.339709 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.339714 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.339717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.339964 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.340826 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:58.340832 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:58.341041 (+ 209us) client_negotiation.cc:769] Sending connection context
0928 16:42:58.341223 (+ 182us) client_negotiation.cc:240] Negotiation successful
0928 16:42:58.341636 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260928 16:42:58.349713 1794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf: Bootstrap starting.
I20260928 16:42:58.355032 1794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:58.356874 1794 log.cc:824] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:58.361382 1794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf: No bootstrap required, opened a new log
I20260928 16:42:58.374054 1794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } }
I20260928 16:42:58.374658 1794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 158e1da36b1241ed9353dbcbc3346ebf, State: Initialized, Role: FOLLOWER
I20260928 16:42:58.375386 1794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } }
I20260928 16:42:58.376879 1892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.377369 1892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:58.378561 1794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [sys.catalog]: configured and running, proceeding with master startup.
W20260928 16:42:58.392367 1903 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:42:58.392688 1903 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:42:58.388327 1726 master_runner.cc:186] Error getting master registration for 127.10.247.254:44475: OK, instance_id { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" instance_seqno: 1790613776994580 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:42:58.401108 1726 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:42:58.428131 1719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } attempt: 5
W20260928 16:42:58.435515 1823 tablet.cc:2369] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 16:42:58 dist-test-slave-9fn6 krb5kdc[1638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790613776, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.247.254@KRBTEST.COM for kudu/127.10.247.252@KRBTEST.COM
I20260928 16:42:58.445834 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.429756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37900 (local address 127.10.247.252:33909)
0928 16:42:58.429971 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.429978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.430001 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:42:58.430081 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.430087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.430213 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.430327 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.431369 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.432630 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.433976 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.434722 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.434848 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.439104 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.439111 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.439114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.439170 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.442795 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.443514 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.443525 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.443529 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.443705 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.444110 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.444115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.444118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.444511 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.444687 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.445120 (+ 433us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.445364 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 16:42:58.445894 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.429124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.247.252:33909 (local address 127.0.0.1:37900)
0928 16:42:58.429350 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0928 16:42:58.429376 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 16:42:58.429535 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:42:58.430406 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:42:58.430412 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:42:58.430933 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:42:58.431160 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.431173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.432783 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.432791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.433800 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:42:58.433817 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.434893 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.434899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:42:58.435102 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.435913 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:42:58.435939 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:42:58.438872 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:42:58.442937 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.442944 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.442949 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.443379 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.443824 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:42:58.443829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:42:58.443832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:42:58.443999 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:42:58.444670 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:42:58.444674 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:42:58.444854 (+ 180us) client_negotiation.cc:769] Sending connection context
0928 16:42:58.444998 (+ 144us) client_negotiation.cc:240] Negotiation successful
0928 16:42:58.445341 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260928 16:42:58.453729 1719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f: Bootstrap starting.
I20260928 16:42:58.458720 1719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f: Neither blocks nor log segments found. Creating new log.
I20260928 16:42:58.460276 1719 log.cc:824] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f: Log is configured to *not* fsync() on all Append() calls
I20260928 16:42:58.464270 1719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f: No bootstrap required, opened a new log
I20260928 16:42:58.472666 1719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } }
I20260928 16:42:58.473225 1719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8394c5357c354b21b27053b3de7fa16f, State: Initialized, Role: FOLLOWER
I20260928 16:42:58.473814 1719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } }
I20260928 16:42:58.475211 1906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.475927 1906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:58.476718 1719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:42:58.476778 1694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 16:42:58.477008 1694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:58.481173 1694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } }
W20260928 16:42:58.483422 1673 tablet.cc:2369] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:58.484715 1769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8394c5357c354b21b27053b3de7fa16f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "158e1da36b1241ed9353dbcbc3346ebf"
I20260928 16:42:58.484882 1844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8394c5357c354b21b27053b3de7fa16f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ef07135ca00942779ea8311e84593dd2"
I20260928 16:42:58.485364 1844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:58.485371 1769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:42:58.489253 1769 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8394c5357c354b21b27053b3de7fa16f in term 1.
I20260928 16:42:58.489253 1844 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8394c5357c354b21b27053b3de7fa16f in term 1.
I20260928 16:42:58.490170 1668 leader_election.cc:302] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8394c5357c354b21b27053b3de7fa16f, ef07135ca00942779ea8311e84593dd2; no voters:
I20260928 16:42:58.490855 1694 leader_election.cc:288] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [CANDIDATE]: Term 1 election: Requested vote from peers 158e1da36b1241ed9353dbcbc3346ebf (127.10.247.253:46561), ef07135ca00942779ea8311e84593dd2 (127.10.247.252:33909)
I20260928 16:42:58.491793 1906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:42:58.515420 1906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [term 1 LEADER]: Becoming Leader. State: Replica: 8394c5357c354b21b27053b3de7fa16f, State: Running, Role: LEADER
I20260928 16:42:58.516388 1906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } }
I20260928 16:42:58.535509 1908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8394c5357c354b21b27053b3de7fa16f. Latest consensus state: current_term: 1 leader_uuid: "8394c5357c354b21b27053b3de7fa16f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.536198 1908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:58.543340 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.513761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.10.247.252:33909)
0928 16:42:58.514027 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.514034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.515276 (+ 1242us) server_negotiation.cc:407] Connection header received
0928 16:42:58.520514 (+ 5238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.520523 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.520676 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.520762 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.523520 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.524705 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.527528 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.528302 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.528421 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.534556 (+ 6135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.534564 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.534569 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.534622 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.538922 (+ 4300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.540368 (+ 1446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.540376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.540381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.540589 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.541205 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.541211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.541215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.541661 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.541950 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.542605 (+ 655us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.542921 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:42:58.547672 1922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:42:58.559110 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.515721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54126 (local address 127.10.247.253:46561)
0928 16:42:58.516008 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.516014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.529952 (+ 13938us) server_negotiation.cc:407] Connection header received
0928 16:42:58.530545 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.530552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.530657 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.530735 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.542448 (+ 11713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.543728 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.545981 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.546740 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.546846 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.550522 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.550530 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.550535 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.550587 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.554279 (+ 3692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.555280 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.555287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.555291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.555545 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.556076 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.556081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.556085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.556773 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.557137 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.558293 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.558586 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260928 16:42:58.564553 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:58.513274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.10.247.254:44475)
0928 16:42:58.513491 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.513500 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.526605 (+ 13105us) server_negotiation.cc:407] Connection header received
0928 16:42:58.526892 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.526900 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.527010 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.527088 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.529906 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.531135 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.544839 (+ 13704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:58.545539 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:58.545654 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:58.554443 (+ 8789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:42:58.554450 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:42:58.554455 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:42:58.554506 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:42:58.558268 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.561727 (+ 3459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.561731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.561734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.561911 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:42:58.562386 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:42:58.562393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:42:58.562398 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:42:58.562903 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:42:58.563219 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:58.563813 (+ 594us) server_negotiation.cc:299] Negotiation successful
0928 16:42:58.564095 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 16:42:58.568485 1922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260928 16:42:58.574038 1748 tablet.cc:2369] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:42:58.597047 1844 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [term 1 FOLLOWER]: Refusing update from remote peer 8394c5357c354b21b27053b3de7fa16f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:58.600667 1908 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:58.603157 1769 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [term 1 FOLLOWER]: Refusing update from remote peer 8394c5357c354b21b27053b3de7fa16f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:42:58.604674 1906 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [LEADER]: Connected to new peer: Peer: permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:42:58.649771 1892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8394c5357c354b21b27053b3de7fa16f. Latest consensus state: current_term: 1 leader_uuid: "8394c5357c354b21b27053b3de7fa16f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.650144 1892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:58.654094 1927 mvcc.cc:205] Tried to move back new op lower bound from 7334354037097406464 to 7334354036812161024. Current Snapshot: MvccSnapshot[applied={T|T < 7334354037097406464}]
I20260928 16:42:58.662788 1878 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8394c5357c354b21b27053b3de7fa16f. Latest consensus state: current_term: 1 leader_uuid: "8394c5357c354b21b27053b3de7fa16f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.663203 1878 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:58.677479 1878 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8394c5357c354b21b27053b3de7fa16f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.677837 1878 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:58.677897 1892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8394c5357c354b21b27053b3de7fa16f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.678220 1892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:42:58.682312 1929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8394c5357c354b21b27053b3de7fa16f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.682818 1929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:58.683301 1929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8394c5357c354b21b27053b3de7fa16f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8394c5357c354b21b27053b3de7fa16f" member_type: VOTER last_known_addr { host: "127.10.247.254" port: 44475 } } peers { permanent_uuid: "158e1da36b1241ed9353dbcbc3346ebf" member_type: VOTER last_known_addr { host: "127.10.247.253" port: 46561 } } peers { permanent_uuid: "ef07135ca00942779ea8311e84593dd2" member_type: VOTER last_known_addr { host: "127.10.247.252" port: 33909 } } }
I20260928 16:42:58.683548 1929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f [sys.catalog]: This master's current role is: LEADER
I20260928 16:42:58.685943 1922 catalog_manager.cc:1347] Generated new cluster ID: a185a6730ab14b15b258d688298b0155
I20260928 16:42:58.686044 1922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:42:58.734589 1922 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:42:58.737151 1922 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:42:58.764837 1922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8394c5357c354b21b27053b3de7fa16f: Generated new TSK 0
I20260928 16:42:58.771852 1922 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 16:42:58.940789 1873 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:58.937344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44942 (local address 127.10.247.254:44475)
0928 16:42:58.937644 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.937650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.938234 (+ 584us) server_negotiation.cc:407] Connection header received
0928 16:42:58.938613 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.938620 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.938738 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.938831 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.940402 (+ 1571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44942: BlockingRecv error: recv got EOF from 127.0.0.1:44942 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260928 16:42:58.944027 1872 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:58.937808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.10.247.252:33909)
0928 16:42:58.938015 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.938022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.941950 (+ 3928us) server_negotiation.cc:407] Connection header received
0928 16:42:58.942265 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.942273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.942384 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.942465 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.943724 (+ 1259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37928: BlockingRecv error: recv got EOF from 127.0.0.1:37928 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260928 16:42:58.946979 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:58.938803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54136 (local address 127.10.247.253:46561)
0928 16:42:58.939037 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.939044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.944981 (+ 5937us) server_negotiation.cc:407] Connection header received
0928 16:42:58.945275 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.945282 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.945409 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.945496 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.946643 (+ 1147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54136: BlockingRecv error: recv got EOF from 127.0.0.1:54136 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260928 16:42:58.956813 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:58.954161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.10.247.253:46561)
0928 16:42:58.954440 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.954447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.954956 (+ 509us) server_negotiation.cc:407] Connection header received
0928 16:42:58.955231 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.955236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.955367 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.955466 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.956456 (+ 990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54144: BlockingRecv error: recv got EOF from 127.0.0.1:54144 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260928 16:42:58.963315 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:58.961359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54152 (local address 127.10.247.253:46561)
0928 16:42:58.961550 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.961556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.961577 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:42:58.961856 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.961862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.961952 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.962058 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.963050 (+ 992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54152: BlockingRecv error: recv got EOF from 127.0.0.1:54152 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260928 16:42:58.971069 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:58.969259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54158 (local address 127.10.247.253:46561)
0928 16:42:58.969448 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.969454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.969560 (+ 106us) server_negotiation.cc:407] Connection header received
0928 16:42:58.969877 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.969883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.969967 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.970058 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.970839 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54158: BlockingRecv error: recv got EOF from 127.0.0.1:54158 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260928 16:42:58.982395 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:58.979801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54162 (local address 127.10.247.253:46561)
0928 16:42:58.980107 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.980113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.980678 (+ 565us) server_negotiation.cc:407] Connection header received
0928 16:42:58.980966 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.980971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.981064 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.981206 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.982121 (+ 915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54162: BlockingRecv error: recv got EOF from 127.0.0.1:54162 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
W20260928 16:42:58.996889 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:58.994282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54172 (local address 127.10.247.253:46561)
0928 16:42:58.994535 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:58.994543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:58.995175 (+ 632us) server_negotiation.cc:407] Connection header received
0928 16:42:58.995409 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:58.995414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:58.995539 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:58.995636 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:58.996485 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54172: BlockingRecv error: recv got EOF from 127.0.0.1:54172 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260928 16:42:59.023420 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:59.020994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.10.247.253:46561)
0928 16:42:59.021247 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:59.021255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:59.021831 (+ 576us) server_negotiation.cc:407] Connection header received
0928 16:42:59.022068 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:59.022073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:59.022165 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:59.022262 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:59.023134 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54176: BlockingRecv error: recv got EOF from 127.0.0.1:54176 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260928 16:42:59.062016 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:59.059670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.10.247.253:46561)
0928 16:42:59.059934 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:59.059942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:59.060243 (+ 301us) server_negotiation.cc:407] Connection header received
0928 16:42:59.060472 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:59.060480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:59.060670 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:59.060818 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:59.061698 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54178: BlockingRecv error: recv got EOF from 127.0.0.1:54178 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260928 16:42:59.137552 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:59.134546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54188 (local address 127.10.247.253:46561)
0928 16:42:59.134799 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:59.134807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:59.135524 (+ 717us) server_negotiation.cc:407] Connection header received
0928 16:42:59.135804 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:59.135811 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:59.135956 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:59.136108 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:59.137239 (+ 1131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54188: BlockingRecv error: recv got EOF from 127.0.0.1:54188 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260928 16:42:59.272414 1875 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:42:59.270080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.10.247.253:46561)
0928 16:42:59.270281 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:59.270287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:59.270648 (+ 361us) server_negotiation.cc:407] Connection header received
0928 16:42:59.270847 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:59.270852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:59.270948 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:59.271102 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:42:59.272105 (+ 1003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54200: BlockingRecv error: recv got EOF from 127.0.0.1:54200 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 16:42:59.318857 1889 catalog_manager.cc:1259] Loaded cluster ID: a185a6730ab14b15b258d688298b0155
I20260928 16:42:59.319017 1889 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2: loading cluster ID for follower catalog manager: success
I20260928 16:42:59.322299 1889 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2: acquiring CA information for follower catalog manager: success
I20260928 16:42:59.324721 1889 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ef07135ca00942779ea8311e84593dd2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:42:59.395325 1903 catalog_manager.cc:1259] Loaded cluster ID: a185a6730ab14b15b258d688298b0155
I20260928 16:42:59.395467 1903 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf: loading cluster ID for follower catalog manager: success
I20260928 16:42:59.398442 1903 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf: acquiring CA information for follower catalog manager: success
I20260928 16:42:59.400571 1903 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 158e1da36b1241ed9353dbcbc3346ebf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:42:59.540709 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:59.533926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54216 (local address 127.10.247.253:46561)
0928 16:42:59.534260 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:59.534265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:59.534435 (+ 170us) server_negotiation.cc:407] Connection header received
0928 16:42:59.534685 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:59.534690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:59.534915 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:59.535062 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:59.535605 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.536377 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.537964 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.538434 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.538544 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:59.539125 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:59.539857 (+ 732us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:59.539984 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:59.540205 (+ 221us) server_negotiation.cc:299] Negotiation successful
0928 16:42:59.540308 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260928 16:42:59.554418 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:59.547943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37936 (local address 127.10.247.252:33909)
0928 16:42:59.548500 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:59.548506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:59.548526 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:42:59.548773 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:59.548778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:59.548985 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:59.549172 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:59.549587 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.550403 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.551836 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.552347 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.552457 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:59.553023 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:59.553737 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:59.553859 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:59.553963 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 16:42:59.554047 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":320,"threads_started":1}
I20260928 16:42:59.572404 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:59.561397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.10.247.252:33909)
0928 16:42:59.564981 (+ 3584us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:59.564988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:59.565011 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:42:59.565103 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:59.565107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:59.565185 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:59.565270 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:59.565959 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.566954 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.569176 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.569833 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.569965 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:59.571522 (+ 1557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:59.571692 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:59.571780 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:59.571951 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 16:42:59.572056 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3432}
I20260928 16:42:59.573535 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:59.561285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.10.247.254:44475)
0928 16:42:59.565796 (+ 4511us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:59.565803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:59.566360 (+ 557us) server_negotiation.cc:407] Connection header received
0928 16:42:59.566535 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:59.566541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:59.566836 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:59.567051 (+ 215us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:59.567815 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.568866 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.570548 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.571211 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.571342 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:59.571843 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:59.572457 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:59.572566 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:59.573047 (+ 481us) server_negotiation.cc:299] Negotiation successful
0928 16:42:59.573200 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4361,"thread_start_us":4223,"threads_started":1}
I20260928 16:42:59.574246 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:42:59.566441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54230 (local address 127.10.247.253:46561)
0928 16:42:59.566667 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 16:42:59.566673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:42:59.567966 (+ 1293us) server_negotiation.cc:407] Connection header received
0928 16:42:59.568151 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:42:59.568157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:42:59.568274 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:42:59.568368 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:42:59.569050 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.570134 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.572081 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:42:59.572796 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:42:59.572890 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:42:59.573452 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:42:59.573565 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:42:59.573649 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 16:42:59.573849 (+ 200us) server_negotiation.cc:299] Negotiation successful
0928 16:42:59.573946 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 16:42:59.579602 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1651
I20260928 16:42:59.580844 1714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:59.708736 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 1651
I20260928 16:42:59.730485 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1726
I20260928 16:42:59.731911 1789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:42:59.856765 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 1726
I20260928 16:42:59.878558 11231 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKneNwT/build/asan/bin/kudu pid 1802
I20260928 16:42:59.879995 1864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:43:00.008891 11231 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKneNwT/build/asan/bin/kudu with pid 1802
2026-09-28T16:43:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4916 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9841 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (391457 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST